Key Takeaways
- Codemods, developed by Facebook, offer a powerful solution for automating large-scale refactoring tasks, enabling developers to efficiently update codebases to align with new JavaScript standards and features.
- JSCodeshift, an advancement over basic codemods, utilizes an Abstract Syntax Tree (AST) for precise, context-aware code transformations across multiple files, simplifying the refactoring process.
- The use of tools like ASTExplorer enhances understanding and manipulation of code structure, facilitating the creation and application of custom codemods tailored to specific refactoring needs.
- Pre-existing codemods available on platforms like GitHub can significantly reduce the effort required in common refactoring scenarios, such as updating variable declarations or migrating to new coding patterns.
- Integrating codemods into the development workflow can not only accelerate the update and maintenance of code but also contribute to more consistent, readable, and robust applications.
Maintaining a codebase can be a frustrating experience for any developer, especially a JavaScript codebase. With ever-changing standards, syntax, and third party package breaking changes, it can be hard to keep up.
In recent years, the JavaScript landscape has changed beyond recognition. Advancements in the core JavaScript language has meant that even the simplest simple task of variable declaration has been changed. ES6 introduced let and const, arrow functions, and many more core changes, each bringing improvements and benefits to developers and their applications. Pressure on developers to produce and maintain code that will stand up to the test of time is on the increase. This article will show you how you can automate large-scale refactoring tasks with the use of codemods and the JSCodeshift tool, allowing you to easily update your code to take advantage of newer language features, for example.Codemod
Codemod is a tool developed by Facebook to help with the refactor of large-scale codebases. It enables the developer to refactor a large codebase in a small amount of time. In some cases, a developer might use an IDE to perform the refactor of a class or variable name, however, this is usually scoped to one file at a time. The next tool in a developer’s refactoring tool kit is a global find and replace. This can work in many cases with the use of complex regular expressions. Many scenarios are not suited to this method; for example, when there are multiple implementations that need to be changed. Codemod is a Python tool that takes a number of parameters including the expression you wish to match and the replacement.codemod -m -d /code/myAwesomeSite/pages --extensions php,html \
'<font *color="?(.*?)"?>(.*?)</font>' \
'<span style="color: \1;">\2</span>'
In the above example, we are replacing the usage of the <font>
tag with a span and inlining the color style. The first two parameters are flags to indicate multiple line matching (-m) and the directory to start processing from (-d /code/myAwesomeSite/pages). We can also restrict the extensions that are processed (–extensions php,html). We then supply the match expression and the replacement. If the replacement is not provided we will be prompted for one at runtime. The tool works, but it is very similar to existing regular expression matching tools.
JSCodeshift
JSCodeshift is the next step up in the refactor toolkit. Also developed by Facebook, its a tool for running codemods across multiple files. As a Node module, JSCodeshift provides a clean and easy-to-use API, and uses Recast under the hood. Recast is an AST-to-AST (Abstract Syntax Tree) transformation tool.Recast
Recast is a Node module that exposes an interface for parsing and reprinting JavaScript code. It can parse code in string format and generates an object from this which follows an AST structure. This allows us to inspect the code for patterns such as a function declarations.var recast = require("recast");
var code = [
"function add(a, b) {",
" return a + b",
"}"
].join("\n");
var ast = recast.parse(code);
console.log(ast);
//output
{
"program": {
"type": "Program",
"body": [
{
"type": "FunctionDeclaration",
"id": {
"type": "Identifier",
"name": "add",
"loc": {
"start": {
"line": 1,
"column": 9
},
"end": {
"line": 1,
"column": 12
},
"lines": {},
"indent": 0
}
},
...........
As we can see from the above example, we pass in the code string for a function that adds two numbers. When we parse and log the object we can see the AST. We see the FunctionDeclaration
and the name of the function etc. As this is just a JavaScript object we can modify it as we see fit. Then we can trigger the print function to return the updated code string.
AST (Abstract Syntax Tree)
As mentioned before, Recast builds an AST from our code string. An AST is a tree representation of the abstract syntax of source code. Each node of the tree represents a construct in the source code and the node provides important information about the construct. ASTExplorer is a browser-based tool that can help to parse and understand the tree of your code. Using ASTExplorer we can view the AST of a simple code example. Starting with our code, we will declare a const called foo and this will equal the string of ‘bar’.const foo = 'bar';
This results in the below AST:
We can see the VariableDeclaration under the body array, which contains our const. All VariableDeclarations have an id attribute that contains our important information such as name etc. If we were building a codemod to rename all instances of foo
we can use this name attribute and iterate over all the instances to change the name.
Installation and Usage
Using the tools and techniques from above we can now fully take advantage of JSCodeshift. As JSCodeshift is a node module we can install it at the project or global level.npm install -g jscodeshift
Once installed, we can use existing codemods with JSCodeshift. We must provide some parameters to tell JSCodeshift what we want to achieve. The basic syntax is calling jscodeshift
with a path of the file or files we wish to transform. The essential parameter is the location of the transform (-t). This can either be a local file or a URL to a codemod file. The transform parameter defaults to look for a transform.js
file in the current directory.
Other useful parameters include dry run (-d), which will apply the transform but not update the files, and Verbose (-v), which will log out all information about the transform process. Transforms are codemods, simple JavaScript modules that export a function. This function accepts the following parameters:
- fileInfo
- api
- options
jscodeshift -t myTransforms fileA fileB --codeVersion=1.2
. Options would then contain {codeVersion: '1.2'}
.
Inside the function we expose, we must return the transformed code as a string. For example if we have the code string of const foo = 'bar'
and we would like to transform it to replace the const foo with const bar, our codemod would look like this:
export default function transformer(file, api) {
const j = api.jscodeshift;
return j(file.source)
.find(j.Identifier)
.forEach(path => {
j(path).replaceWith(
j.identifier('bar')
);
})
.toSource();
}
As you can see, we chain a number of functions together and call toSource()
at the end to generate the transformed code string.
There are some rules we must follow when returning the code. Returning a string that is different to the input will trigger a successful transform. If the string is the same as the input then the transform will be unsuccessful and if nothing is returned then the transform will not be necessary. JSCodeshift then uses these results when processing stats on the transforms.
Existing codemods
In most cases, developers will not need to write their own codemod. Many common refactoring actions have already been turned into codemods. Some examples include js-codemod no-vars which will convert all instances ofvar
into either let
or const
, based on the variable usage. For example, let if the variable is reassigned at a later time and const when the variable is never reassigned.
js-codemod template-literals will replace instances of string concatenation with template literals e.g.
const sayHello = 'Hi my name is ' + name;
//after transform
const sayHello = `Hi my name is ${name}`;
How codemods are written
We can take the no-vars codemod from above and break down the code to see how a complex codemod works.const updatedAnything = root.find(j.VariableDeclaration).filter(
dec => dec.value.kind === 'var'
).filter(declaration => {
return declaration.value.declarations.every(declarator => {
return !isTruelyVar(declaration, declarator);
});
}).forEach(declaration => {
const forLoopWithoutInit = isForLoopDeclarationWithoutInit(declaration);
if (
declaration.value.declarations.some(declarator => {
return (!declarator.init && !forLoopWithoutInit) || isMutated(declaration, declarator);
})
) {
declaration.value.kind = 'let';
} else {
declaration.value.kind = 'const';
}
}).size() !== 0;
return updatedAnything ? root.toSource() : null;
The above code is the core of the no-vars codemod. First, a filter is run on all VariableDeclaration’s this includes var, let, and const. The filter only returns var declarations. Which are passed into a second filter, this calls the custom function isTruelyVar
. This is used to determine the nature of the var (e.g is the var inside a closure or declared twice or is a function declaration which might be hoisted). This will determine if is safe to do the conversion on the var. For each var that passes the isTruelyVar
filter, they are processed in a forEach loop.
Inside the loop, a check is made on the var, if the var is inside a loop e.g.
for(var i = 0; i < 10; i++) {
doSomething();
}
To detect if the var is inside a loop the parent type can be checked.
const isForLoopDeclarationWithoutInit = declaration => {
const parentType = declaration.parentPath.value.type;
return parentType === 'ForOfStatement' || parentType === 'ForInStatement';
};
If the var is inside a loop and is not mutated then is can be changed to a const. Checking for mutations can be done by filtering over the var nodes AssignmentExpression’s and UpdateExpression’s. AssignmentExpression will show where and when the var was assigned to e.g
var foo = 'bar';
UpdateExpression will show where and when the var was updated e.g.
var foo = 'bar';
foo = 'Foo Bar'; //Updated
If the var is is inside a loop with mutation then a let is used as let can be reassigned after being instantiated. The last line the in codemod checked if anything was updated e.g. any var’s were changed. If so the new source of the file is returned else null is returned, which tells JSCodeshift that no processing was done. The full source for the codemod can be found here.
The Facebook team have also added a number of codemods for updating React syntax and to handle changes to the React API. Some codemods include react-codemod sort-comp which sorts React lifecycle methods to match the ESlint sort-comp rule.
The most recent and popular React codemod is React-PropTypes-to-prop-types which helps in the recent change from the core React team to move React.PropTypes into its own node module. This means from React v16, developers will need to install prop-types if they wish to continue using propTypes in components. This is a great example of the use case for a codemod. The method of using PropTypes is not set in stone.
The following are all valid:
Importing React and accessing PropTypes from the default import:
import React from 'react';
class HelloWorld extends React.Component {
static propTypes = {
name: React.PropTypes.string,
}
.....
Importing React and the named import for PropTypes:
import React, { PropTypes, Component } from 'react';
class HelloWorld extends Component {
static propTypes = {
name: PropTypes.string,
}
.....
Importing React and the named import for PropTypes but declaring PropTypes on a stateless component:
import React, { PropTypes } from 'react';
const HelloWorld = ({name}) => {
.....
}
HelloWorld.propTypes = {
name: PropTypes.string
};
Having the three ways to implement the same solution makes it especially hard to perform a regular expression to find and replace. If we had the above three in our code base we could easily upgrade to the new PropTypes pattern by running the following:
jscodeshift src/ -t transforms/proptypes.js
In this example, we pulled the PropTypes codemod from the react-codemods repo and added it to a transforms directory in our project. The codemod will add import PropTypes from 'prop-types';
to each file and replace any instances of React.PropTypes
with PropTypes
.
Conclusion
Facebook have pioneered in code maintenance enabling developers to adjust with their ever changing API and code practices. JavaScript fatigue has become a large problem and as I have shown, having tools that can help with the stress of updating existing code can assist towards the reduce of this fatigue. In the world of server-side development with database reliance, developers regularly create migration scripts to maintain database support and to ensure users are up to date with the latest version of their database. JavaScript library maintainers could provide codemods as a migration script when major versions are released, with breaking changes a codemod could handle the upgrade process. This would fit into the existing migration process as with npm install’s scripts can be run. Having a codemod run automatically at install/upgrade time could speed up upgrades and provide more confidence in the consumer. Including this into the release process would be beneficial for not just consumers but also reduce overhead for maintainers when updating examples and guides. In this article, we have seen the powerful nature of codemods and JSCodeshift and how they can quickly update complex code. From the beginning with the Codemod tool and moving on to tools such as ASTExplorer and JSCodeshift we can now build codemods to suit our own needs. Taking advantage of the already wide range of pre-made codemods allows developers to advance in time with the masses. Have you used codemods yet? What is in your toolkit? What other refactors would be a great use for codemods? Let me know in the comments! This article was peer reviewed by Graham Cox and Michael Wanyoike. Thanks to all of SitePoint’s peer reviewers for making SitePoint content the best it can be!Frequently Asked Questions (FAQs) about Codemods
What are the key benefits of using Codemods?
Codemods are a powerful tool for large-scale codebase refactoring. They automate the process of making systematic changes to your code, saving you time and reducing the risk of human error. Codemods can also help to maintain code consistency and improve code readability, which is particularly beneficial in large projects with multiple contributors. They can be used to upgrade dependencies, migrate to a new coding standard, or even to rewrite your code in a different language.
How do I get started with Codemods?
To get started with Codemods, you first need to install the necessary tools. This typically involves installing Node.js and npm (Node Package Manager), and then installing the jscodeshift package, which is the JavaScript codemod toolkit. Once you have these tools installed, you can start writing your own codemods using the jscodeshift API. There are also many pre-written codemods available on GitHub that you can use as a starting point.
Can Codemods handle complex code transformations?
Yes, Codemods can handle complex code transformations. They use Abstract Syntax Trees (ASTs) to understand the structure of your code, which allows them to make precise, context-aware changes. This means they can handle complex transformations that would be difficult or impossible to achieve with simple find-and-replace operations. However, writing a codemod for a complex transformation can be a challenging task and may require a deep understanding of ASTs and the jscodeshift API.
Are there any risks or downsides to using Codemods?
While Codemods are a powerful tool, they are not without their risks. One potential downside is that they can introduce bugs into your code if not used correctly. This is particularly true for complex transformations, where a small mistake in your codemod can have widespread effects. It’s also worth noting that while codemods can save you time in the long run, writing a codemod can be a time-consuming task, particularly if you’re new to the process.
How can I test my Codemods to ensure they’re working correctly?
Testing is a crucial part of the codemod development process. You can use the jscodeshift’s dry-run option to see what changes your codemod would make without actually applying them. This can help you catch any errors or unexpected behavior. Additionally, you should always test your codemod on a small, representative sample of your codebase before applying it to the whole thing. This can help you catch any issues early and minimize the potential damage if something goes wrong.
Can I use Codemods with any programming language?
While the concept of codemods is not specific to any particular programming language, the tools and libraries available for writing and running codemods can vary between languages. The jscodeshift toolkit, for example, is specifically designed for JavaScript. However, there are similar tools available for other languages, such as Coccinelle for C and Comby for a variety of languages including Python, Java, and Go.
What is the role of Abstract Syntax Trees (ASTs) in Codemods?
Abstract Syntax Trees (ASTs) play a crucial role in codemods. They provide a structured representation of your code, which allows codemods to understand the context and semantics of the code they’re modifying. This is what enables codemods to make precise, context-aware changes to your code, and is what sets them apart from simpler find-and-replace tools.
Can Codemods help me migrate to a new coding standard or style guide?
Yes, one of the common uses of codemods is to help teams migrate to a new coding standard or style guide. By automating the process of updating your code to comply with the new standard, codemods can save you a significant amount of time and effort. They can also help to ensure consistency across your codebase, which can make your code easier to read and maintain.
How can I learn more about writing my own Codemods?
There are many resources available online to help you learn more about writing codemods. The jscodeshift GitHub page is a good place to start, as it provides a comprehensive guide to the jscodeshift API. There are also many tutorials and blog posts available that walk you through the process of writing a codemod, and you can find many examples of pre-written codemods on GitHub.
Can Codemods be used to upgrade dependencies in my project?
Yes, codemods can be used to automate the process of upgrading dependencies in your project. This can be particularly useful when a dependency introduces breaking changes, as it allows you to update your code to work with the new version of the dependency without having to manually update every instance where it’s used. However, writing a codemod for this purpose can be complex and requires a good understanding of both the dependency and the jscodeshift API.
Application developer based in Belfast, Northern Ireland. Focused on front end development especially JavaScript. Been working in software development since 2010 and still learning and sharing everyday.