astx
structural search and replace for JavaScript and TypeScript, using jscodeshift
Table of Contents
- astx
- Table of Contents
- Introduction
- Other usage examples
- Prior art and philosophy
- API
- Match Patterns
Introduction
If you've ever refactored a function, and had to go through and change all the calls to that function by hand one by one, you know
how much time it can take. For example, let's say you decided to move an optional boolean force argument to your rmdir function
into an options hash argument:
// before:
rmdir('old/stuff')
rmdir('new/stuff', true)
// after:
rmdir('old/stuff')
rmdir('new/stuff', { force: true })Changing a bunch of calls to rmdir by hand would suck. You could try using regex replace, but it's fiddly and wouldn't tolerate whitespace and
linebreaks well unless you work really hard at the regex.
Now there's a better option...you can refactor with confidence using astx!
// astx.js
astx.find`rmdir($path, $force)`.replace`rmdir($path, { force: $force })`What's going on here? Find and replace must be valid JS expressions or statements. astx parses them
into AST (Abstract Syntax Tree) nodes, and then looks for matching AST nodes in your code.
astx treats any identifier in starting with $ in the find or replace expression as a placeholder - in this case, $path and $force.
(You can use $$ as an escape, for instance $$foo will match literal identifier $foo in your code).
When it gets to a function call, it checks that the function name matches rmdir, and that it has the same number of arguments.
Then it checks if the arguments match.
Our patterns for both arguments ($path, $force) are placeholders, so they automatically match and capture the corresponding AST nodes
of the two arguments in your code.
Then astx replaces that function call it found with the replacement expression. When it finds placeholders in the replacement expression,
it substitutes the corresponding values that were captured for those placeholders ($path captured 'new/stuff' and $force captured true).
Other usage examples
Fixing eslint errors that eslint is too dumb to fix for you
Got a lot of Do not access Object.prototype method 'hasOwnProperty' from target object errors?
exports.find = `$a.hasOwnProperty($b)`
exports.replace = `Object.prototype.hasOwnProperty.call($a, $b)`Prior art and philosophy
While I was thinking about making this I discovered grasp, a similar tool that inspired the $ capture syntax.
There are several reasons I decided to make astx anyway:
- Grasp uses the Acorn parser, which doesn't support TypeScript or Flow code AFAIK
- Hasn't been updated in 4 years
- Grasp's replace pattern syntax is clunkier, placeholders don't match the find pattern syntax:
grasp -e 'setValue($k, $v, true)' -R 'setValueSilently({{k}}, {{v}})' file.js - It has its own DSL (SQuery) that's pretty limited and has a slight learning curve
- I wanted to leverage the power of jscodeshift for advanced use cases that are probably awkward/impossible in Grasp
So the philosophy of astx is:
- Use jscodeshift (and recast) as a solid foundation
- Provide a simple find and replace API that's ideal for simple cases and has minimum learning curve
- Use javascript + jscodeshift for anything more complex, so that you have unlimited flexibility
Jscodeshift has a learning curve, but it's worth learning if you want to do any nontrivial codemods. Paste your code into AST Explorer if you need to learn about the structure of the AST.
API
Note: the identifier j in all code examples is an instance of jscodeshift, as per convention.
class Astx
import { Astx } from 'astx'
import j from 'jscodeshift'
const astx = new Astx(j, j('your code here'))constructor(jscodeshift: JSCodeshift, root: Collection)
jscodeshift must be configured with your desired parser for methods to work correctly.
For instance, if you're using TypeScript, it could be require('jscodeshift').withParser('ts').
root is the JSCodeshift Collection you want to operate on.
.on(root: Collection)
Returns a different Astx instance for the given root. Use this if you want to filter down which nodes to operate on.
.find()
Finds matches for the given pattern within root, and returns a MatchArray containing the matches.
There are several different ways you can call .find:
.find`pattern`(options?: FindOptions).find(pattern: string, options?: FindOptions).find(pattern: ASTNode, options?: FindOptions)
If you give the pattern as a string, it must be a valid expression or statement as parsed by the jscodeshift instance. Otherwise it should be a valid
AST node you already parsed or constructed.
You can interpolate AST nodes in the tagged template literal; it uses jscodeshift.template.expression or jscodeshift.template.statement under the hood.
For example you could do astx.find`${j.identifier('foo')} + 3`()
.find().replace()
Finds and replaces matches for the given pattern within root.
There are several different ways you can call .replace. Note that you can omit the () after .find`pattern` if you're calling .replace.
And you can call .find in any way described above in place of .find`pattern`.
.find`pattern`.replace`replacement`.find`pattern`.replace(replacement: string).find`pattern`.replace(replacement: ASTNode).find`pattern`.replace(replacement: (match: Match<any>, parse: ParseTag) => string).find`pattern`.replace(replacement: (match: Match<any>, parse: ParseTag) => ASTNode)
If you give the replacement as a string, it must be a valid expression or statement as parsed by the jscodeshift instance.
You can give the replacement as an AST node you already parsed or constructed.
Or you can give a replacement function, which will be called with each match and must return a string or ASTNode (you can use the parse tagged template string function provided as the second argument to parse code into a string
via jscodeshift.template.expression or jscodeshift.template.statement).
For example, you could uppercase the function names in all zero-argument function calls (foo(); bar() becomes FOO(); BAR()) with this:
astx
.find`$fn()`
.replace(({ captures: { $fn } }) => `${$fn.name.toUpperCase()}()`)
.findStatements()
Finds matches for the given multi-statement pattern within root, and returns a StatementsMatchArray containing the matches.
There are several different ways you can call .findStatements:
.findStatements`pattern`(options?: FindOptions).findStatements(pattern: string, options?: FindOptions).findStatements(pattern: ASTNode, options?: FindOptions)
If you give the pattern as a string, it must be code for valid statement(s) as parsed by the jscodeshift instance. Otherwise it should be
a valid array of statement AST nodes you already parsed or constructed.
You can interpolate AST nodes in the tagged template literal; it uses jscodeshift.template.statements under the hood.
For example you could do:
astx.findStatements`
const $a = $b;
$_c;
const $d = $a + $e;
`()This would match (for example) the statements const foo = 1; const bar = foo + 5;, with any number of statements between them.
.findStatements().replace()
Finds and replaces matches for the given multi-statement pattern within root.
There are several different ways you can call .replace. Note that you can omit the () after .findStatements`pattern` if you're calling .replace.
And you can call .findStatements in any way described above in place of .findStatements`pattern`.
.findStatements`pattern`.replace`replacement`.findStatements`pattern`.replace(replacement: string).findStatements`pattern`.replace(replacement: Statement | Statement[]).findStatements`pattern`.replace(replacement: (match: Match<any>, parse: ParseTag) => string).findStatements`pattern`.replace(replacement: (match: Match<any>, parse: ParseTag) => Statement | Statement[])
If you give the replacement as a string, it must be valid code for statements as parsed by the jscodeshift instance.
You can give the replacement as statement AST node(s) you already parsed or constructed.
Or you can give a replacement function, which will be called with each match and must return a string, Statement, or array of Statements (you can use the parse tagged template string function provided as the second argument to parse code into a string
via jscodeshift.template.statements).
Match
.path
The ASTPath of the matched node.
.node
The matched ASTNode.
.captures
The ASTNodes captured from placeholders in the match pattern. For example if the pattern was foo($bar), .captures.$bar will be the ASTNode of the first argument.
.pathCaptures
The ASTPaths captured from placeholders in the match pattern. For example if the pattern was foo($bar), .pathCaptures.$bar will be the ASTPath of the first argument.
.arrayCaptures
The ASTNode[]s captured from array placeholders in the match pattern. For example if the pattern was foo({ ...$bar }), .arrayCaptures.$bar will be the ASTNode[]s of the object properties.
.arrayPathCaptures
The ASTPath[]s captured from array placeholders in the match pattern. For example if the pattern was foo({ ...$bar }), .pathArrayCaptures.$bar will be the ASTPath[]s of the object properties.
class MatchArray
Returned by .find(). Just an array of Matches plus the .replace() method.
Match Patterns
Object Matching
An ObjectExpression (aka object literal) pattern will match any ObjectExpression in your code with the same properties in any order.
It will not match if there are missing or additional properties. For example, { foo: 1, bar: $bar } will match { foo: 1, bar: 2 } or { bar: 'hello', foo: 1 }
but not { foo: 1 } or { foo: 1, bar: 2, baz: 3 }.
You can match additional properties by using ...$captureName, for example { foo: 1, ...$rest } will match { foo: 1 }, { foo: 1, bar: 2 }, { foo: 1, bar: 2, ...props } etc.
The additional properties will be captured in match.arrayCaptures/match.arrayPathCaptures, and can be spread in replacement expressions. For example,
astx.find`{ foo: 1, ...$rest }`.replace`{ bar: 1, ...$rest }` will transform { foo: 1, qux: {}, ...props } into { bar: 1, qux: {}, ...props }.
A spread property that isn't of the form /^\$[a-z0-9]+$/i is not a capture variable, for example { ...foo } will only match { ...foo } and { ...$$foo } will only
match { ...$foo } (leading $$ is an escape for $).
There is currently no way to match properties in a specific order, but it could be added in the future.
List Matching
In many cases where there is a list of nodes in the AST you can match
multiple elements with a capture variable starting with $_. For example, [$_before, 3, $_after] will match any array expression containing an element 3; elements before the
first 3 will be captured in $_before and elements after the first 3 will be captured in $_after.
This works even with block statements. For example, function foo() { $_before; throw new Error('test'); $_after; } will match function foo() that contains a throw new Error('test'),
and the statements before and after that throw statement will get captured in $_before and $_after, respectively.
Support Table
Some items marked TODO probably actually work, but are untested.
| Type | Supports list matching? | Notes |
|---|---|---|
ArrayExpression.elements |
✅ | |
ArrayPattern.elements |
✅ | |
BlockStatement.body |
✅ | |
CallExpression.arguments |
✅ | |
Class(Declaration/Expression).implements |
✅ | |
ClassBody.body |
✅ | |
ComprehensionExpression.blocks |
TODO | |
DeclareClass.body |
TODO | |
DeclareClass.implements |
TODO | |
DeclareExportDeclaration.specifiers |
TODO | |
DeclareInterface.body |
TODO | |
DeclareInterface.extends |
TODO | |
DoExpression.body |
TODO | |
ExportNamedDeclaration.specifiers |
✅ | |
Function.decorators |
TODO | |
Function.params |
✅ | |
FunctionTypeAnnotation/TSFunctionType.params |
✅ | |
GeneratorExpression.blocks |
TODO | |
ImportDeclaration.specifiers |
✅ | |
(TS)InterfaceDeclaration.body |
TODO | |
(TS)InterfaceDeclaration.extends |
TODO | |
IntersectionTypeAnnotation/TSIntersectionType.types |
✅ | |
JSX(Element/Fragment).children |
✅ | |
JSX(Opening)Element.attributes |
✅ | |
MethodDefinition.decorators |
TODO | |
NewExpression.arguments |
✅ | |
ObjectExpression.properties |
✅ | |
ObjectPattern.decorators |
TODO | |
ObjectPattern.properties |
✅ | |
(ObjectTypeAnnotation/TSTypeLiteral).properties |
✅ | Use $a: any to match one property, $_a: any to match multiple |
Program.body |
✅ | |
Property.decorators |
TODO | |
SequenceExpression |
✅ | |
SwitchCase.consequent |
✅ | |
SwitchStatement.cases |
TODO | |
TemplateLiteral.quasis/expressions |
❓ not sure if I can come up with a syntax | |
TryStatement.guardedHandlers |
TODO | |
TryStatement.handlers |
TODO | |
TSFunctionType.parameters |
✅ | |
TSCallSignatureDeclaration.parameters |
TODO | |
TSConstructorType.parameters |
TODO | |
TSConstructSignatureDeclaration.parameters |
TODO | |
TSDeclareFunction.params |
TODO | |
TSDeclareMethod.params |
TODO | |
TSEnumDeclaration.members |
TODO | |
TSIndexSignature.parameters |
TODO | |
TSMethodSignature.parameters |
TODO | |
TSModuleBlock.body |
TODO | |
TSTypeLiteral.members |
✅ | |
TupleTypeAnnotation/TSTupleType.types |
✅ | |
(TS)TypeParameterDeclaration |
✅ | |
(TS)TypeParameterInstantiation |
✅ | |
UnionTypeAnnotation/TSUnionType.types |
✅ | |
VariableDeclaration.declarations |
✅ | |
WithStatement.body |
❌ who uses with statements... |
| Backreferences
If you use the same capture variable more than once, subsequent positions will have to match what was captured for the first occurrence of the variable.
For example, the pattern foo($a, $a, $b, $b) will match only foo(1, 1, {foo: 1}, {foo: 1}) in the following:
foo(1, 1, { foo: 1 }, { foo: 1 }) // match
foo(1, 2, { foo: 1 }, { foo: 1 }) // no match
foo(1, 1, { foo: 1 }, { bar: 1 }) // no match