Refactor Code in Your Lunch Break: Getting Started with Codemods

Maintaining a codebase can be a frustrating experience for any developer, especially a JavaScript codebase. With ever-changing standards, syntax, and third party package breaking changes, it can be hard to keep up.

In recent years, the JavaScript landscape has changed beyond recognition. Advancements in the core JavaScript language has meant that even the simplest simple task of variable declaration has been changed. ES6 introduced let and const, arrow functions, and many more core changes, each bringing improvements and benefits to developers and their applications. Pressure on developers to produce and maintain code that will stand up to the test of time is on the increase. This article will show you how you can automate large-scale refactoring tasks with the use of codemods and the JSCodeshift tool, allowing you to easily update your code to take advantage of newer language features, for example.

Codemod

Codemod is a tool developed by Facebook to help with the refactor of large-scale codebases. It enables the developer to refactor a large codebase in a small amount of time. In some cases, a developer might use an IDE to perform the refactor of a class or variable name, however, this is usually scoped to one file at a time. The next tool in a developer’s refactoring tool kit is a global find and replace. This can work in many cases with the use of complex regular expressions. Many scenarios are not suited to this method; for example, when there are multiple implementations that need to be changed. Codemod is a Python tool that takes a number of parameters including the expression you wish to match and the replacement.

codemod -m -d /code/myAwesomeSite/pages --extensions php,html \
    '<font *color="?(.*?)"?>(.*?)</font>' \
    '<span style="color: \1;">\2</span>'

In the above example, we are replacing the usage of the <font> tag with a span and inlining the color style. The first two parameters are flags to indicate multiple line matching (-m) and the directory to start processing from (-d /code/myAwesomeSite/pages). We can also restrict the extensions that are processed (–extensions php,html). We then supply the match expression and the replacement. If the replacement is not provided we will be prompted for one at runtime. The tool works, but it is very similar to existing regular expression matching tools.

JSCodeshift

JSCodeshift is the next step up in the refactor toolkit. Also developed by Facebook, its a tool for running codemods across multiple files. As a Node module, JSCodeshift provides a clean and easy-to-use API, and uses Recast under the hood. Recast is an AST-to-AST (Abstract Syntax Tree) transformation tool.

Recast

Recast is a Node module that exposes an interface for parsing and reprinting JavaScript code. It can parse code in string format and generates an object from this which follows an AST structure. This allows us to inspect the code for patterns such as a function declarations.

var recast = require("recast");

var code = [
    "function add(a, b) {",
    "  return a + b",
    "}"
].join("\n");

var ast = recast.parse(code);
console.log(ast);
//output
{
    "program": {
        "type": "Program",
        "body": [
            {
                "type": "FunctionDeclaration",
                "id": {
                    "type": "Identifier",
                    "name": "add",
                    "loc": {
                        "start": {
                            "line": 1,
                            "column": 9
                        },
                        "end": {
                            "line": 1,
                            "column": 12
                        },
                        "lines": {},
                        "indent": 0
                    }
                },
        ...........    

As we can see from the above example, we pass in the code string for a function that adds two numbers. When we parse and log the object we can see the AST. We see the FunctionDeclaration and the name of the function etc. As this is just a JavaScript object we can modify it as we see fit. Then we can trigger the print function to return the updated code string.

AST (Abstract Syntax Tree)

As mentioned before, Recast builds an AST from our code string. An AST is a tree representation of the abstract syntax of source code. Each node of the tree represents a construct in the source code and the node provides important information about the construct. ASTExplorer is a browser-based tool that can help to parse and understand the tree of your code. Using ASTExplorer we can view the AST of a simple code example. Starting with our code, we will declare a const called foo and this will equal the string of ‘bar’.

const foo = 'bar';

This results in the below AST: We can see the VariableDeclaration under the body array, which contains our const. All VariableDeclarations have an id attribute that contains our important information such as name etc. If we were building a codemod to rename all instances of foo we can use this name attribute and iterate over all the instances to change the name.

Installation and Usage

Using the tools and techniques from above we can now fully take advantage of JSCodeshift. As JSCodeshift is a node module we can install it at the project or global level.

npm install -g jscodeshift

Once installed, we can use existing codemods with JSCodeshift. We must provide some parameters to tell JSCodeshift what we want to achieve. The basic syntax is calling jscodeshift with a path of the file or files we wish to transform. The essential parameter is the location of the transform (-t). This can either be a local file or a URL to a codemod file. The transform parameter defaults to look for a transform.js file in the current directory. Other useful parameters include dry run (-d), which will apply the transform but not update the files, and Verbose (-v), which will log out all information about the transform process. Transforms are codemods, simple JavaScript modules that export a function. This function accepts the following parameters:

• fileInfo
• api
• options

FileInfo holds all the information about the file being currently processed, including path and source. Api is an object that provides access to the JSCodeshift helper functions such as findVariableDeclarators and renameTo. Our last parameter is options, which allows us to pass options from the CLI through to the codemod. For example, if we were running on a deployment server and wanted to add the code version to all files, we could pass it via the CLI jscodeshift -t myTransforms fileA fileB --codeVersion=1.2. Options would then contain {codeVersion: '1.2'}. Inside the function we expose, we must return the transformed code as a string. For example if we have the code string of const foo = 'bar' and we would like to transform it to replace the const foo with const bar, our codemod would look like this:

export default function transformer(file, api) {
  const j = api.jscodeshift;

  return j(file.source)
    .find(j.Identifier)
    .forEach(path => {
      j(path).replaceWith(
        j.identifier('bar')
      );
    })
    .toSource();
}

As you can see, we chain a number of functions together and call toSource() at the end to generate the transformed code string. There are some rules we must follow when returning the code. Returning a string that is different to the input will trigger a successful transform. If the string is the same as the input then the transform will be unsuccessful and if nothing is returned then the transform will not be necessary. JSCodeshift then uses these results when processing stats on the transforms.

Existing codemods

In most cases, developers will not need to write their own codemod. Many common refactoring actions have already been turned into codemods. Some examples include js-codemod no-vars which will convert all instances of var into either let or const, based on the variable usage. For example, let if the variable is reassigned at a later time and const when the variable is never reassigned. js-codemod template-literals will replace instances of string concatenation with template literals e.g.

const sayHello = 'Hi my name is ' + name;
//after transform
const sayHello = `Hi my name is ${name}`;

How codemods are written

We can take the no-vars codemod from above and break down the code to see how a complex codemod works.

const updatedAnything = root.find(j.VariableDeclaration).filter(
            dec => dec.value.kind === 'var'
        ).filter(declaration => {
            return declaration.value.declarations.every(declarator => {
                return !isTruelyVar(declaration, declarator);
            });
        }).forEach(declaration => {
            const forLoopWithoutInit = isForLoopDeclarationWithoutInit(declaration);
            if (
                declaration.value.declarations.some(declarator => {
                    return (!declarator.init && !forLoopWithoutInit) || isMutated(declaration, declarator);
                })
            ) {
                declaration.value.kind = 'let';
            } else {
                declaration.value.kind = 'const';
            }
        }).size() !== 0;
    return updatedAnything ? root.toSource() : null;

The above code is the core of the no-vars codemod. First, a filter is run on all VariableDeclaration’s this includes var, let, and const. The filter only returns var declarations. Which are passed into a second filter, this calls the custom function isTruelyVar. This is used to determine the nature of the var (e.g is the var inside a closure or declared twice or is a function declaration which might be hoisted). This will determine if is safe to do the conversion on the var. For each var that passes the isTruelyVar filter, they are processed in a forEach loop. Inside the loop, a check is made on the var, if the var is inside a loop e.g.

for(var i = 0; i < 10; i++) {
    doSomething();
}

To detect if the var is inside a loop the parent type can be checked.

const isForLoopDeclarationWithoutInit = declaration => {
        const parentType = declaration.parentPath.value.type;
        return parentType === 'ForOfStatement' || parentType === 'ForInStatement';
    };

If the var is inside a loop and is not mutated then is can be changed to a const. Checking for mutations can be done by filtering over the var nodes AssignmentExpression’s and UpdateExpression’s. AssignmentExpression will show where and when the var was assigned to e.g

var foo = 'bar';

UpdateExpression will show where and when the var was updated e.g.

var foo = 'bar';
foo = 'Foo Bar'; //Updated

If the var is is inside a loop with mutation then a let is used as let can be reassigned after being instantiated. The last line the in codemod checked if anything was updated e.g. any var’s were changed. If so the new source of the file is returned else null is returned, which tells JSCodeshift that no processing was done. The full source for the codemod can be found here. The Facebook team have also added a number of codemods for updating React syntax and to handle changes to the React API. Some codemods include react-codemod sort-comp which sorts React lifecycle methods to match the ESlint sort-comp rule. The most recent and popular React codemod is React-PropTypes-to-prop-types which helps in the recent change from the core React team to move React.PropTypes into its own node module. This means from React v16, developers will need to install prop-types if they wish to continue using propTypes in components. This is a great example of the use case for a codemod. The method of using PropTypes is not set in stone. The following are all valid: Importing React and accessing PropTypes from the default import:

import React from 'react';

class HelloWorld extends React.Component {

    static propTypes = {
        name: React.PropTypes.string,
    }
    .....

Importing React and the named import for PropTypes:

import React, { PropTypes, Component } from 'react';

class HelloWorld extends Component {

    static propTypes = {
        name: PropTypes.string,
    }
    .....

Importing React and the named import for PropTypes but declaring PropTypes on a stateless component:

import React, { PropTypes } from 'react';

const HelloWorld = ({name}) => {
    .....
}

HelloWorld.propTypes = {
    name: PropTypes.string
};

Having the three ways to implement the same solution makes it especially hard to perform a regular expression to find and replace. If we had the above three in our code base we could easily upgrade to the new PropTypes pattern by running the following:

jscodeshift src/ -t transforms/proptypes.js

In this example, we pulled the PropTypes codemod from the react-codemods repo and added it to a transforms directory in our project. The codemod will add import PropTypes from 'prop-types'; to each file and replace any instances of React.PropTypes with PropTypes.

Conclusion

Facebook have pioneered in code maintenance enabling developers to adjust with their ever changing API and code practices. JavaScript fatigue has become a large problem and as I have shown, having tools that can help with the stress of updating existing code can assist towards the reduce of this fatigue. In the world of server-side development with database reliance, developers regularly create migration scripts to maintain database support and to ensure users are up to date with the latest version of their database. JavaScript library maintainers could provide codemods as a migration script when major versions are released, with breaking changes a codemod could handle the upgrade process. This would fit into the existing migration process as with npm install’s scripts can be run. Having a codemod run automatically at install/upgrade time could speed up upgrades and provide more confidence in the consumer. Including this into the release process would be beneficial for not just consumers but also reduce overhead for maintainers when updating examples and guides. In this article, we have seen the powerful nature of codemods and JSCodeshift and how they can quickly update complex code. From the beginning with the Codemod tool and moving on to tools such as ASTExplorer and JSCodeshift we can now build codemods to suit our own needs. Taking advantage of the already wide range of pre-made codemods allows developers to advance in time with the masses. Have you used codemods yet? What is in your toolkit? What other refactors would be a great use for codemods? Let me know in the comments! This article was peer reviewed by Graham Cox and Michael Wanyoike. Thanks to all of SitePoint’s peer reviewers for making SitePoint content the best it can be!

Frequently Asked Questions (FAQs) about Codemods