Call sub program before replacing matches

[stefanbuck]

Hey 👋

first of all comby is a really amazing project and the concept behind blow my mind away. My use case is quite advanced and goes beyond substitution so i'm curious what you think about it. Let's imagine the following JavaScript code.

function foo() {}

and let's say I want to be able to rewrite this to

function oof() {}

or even more advanced to

function oOf() {}

I was thinking, wouldn't it be neat if comby would be able to call out another sub program for each match? In the example above the match foo gets passed to another script e.g. bash script which does something with the string and returns the final value.

Concret example: I want to use comby to add i18n to an existing code base. Therefore I need to first find all the texts (which is quite easy with comby) and replace the text with a i18n-key and also write this key/value pair to a dictionary.

How does this sound and does this match with the project vision?

[rvantonder]

Hi @stefanbuck, happy you find comby useful.

Built-in scripting supporting is something I've thought about, but only briefly. For example, an existing tool called Coccinelle, for rewriting C code, integrates Python scripting (example 1, example2). So this approach has been valuable in related tools. The main thing I wonder about (and have not thought through entirely) is what the big advantages are of native supporting scripting in comby (like the workflow you mention), versus an API that allows people to script on top of comby.

For example, in your concrete case, another way of performing the same thing by scripting "on top of" comby would be to create, say, a Python script that:

1. Gets all the matches in JSON format (this can be dumped to file, or read in directly using Python bindings)
2. Perform the arbitrary scripting (write the values into the dictionary)
3. Replace contents in files (substitute the matches).

Obviously 2. and 3. don't have to happen in that order. comby has support for dumping the JSON format, and substituting the values, so this is all possible today. If you are comfortable with Python, you can try out the bindings that have support for getting matches/replacing/substituting, etc., here: https://github.com/ChrisTimperley/comby-python

---

So yes, once there is a clear idea that having native support is a big win over scripting on top of the primitive operations, adding scripting would be a nice thing to have. For now there haven't been too many requests for this, and it would need some careful thinking/design. Let me know if you have more thoughts on that :)

[stefanbuck]

Thanks for your quick reply. Yeah I thought about scripting on top of comby as well. However, it feels like there is just one tiny piece missing (at least for my use case). I wasn't necessarily talking about adding scripting support. My idea was more something in this direction

comby ... -substitution-handler=./custom.js

-subsitiation-handler can be any executable. In this case it's a simple NodeJS script. Comby calls the substitution handler with the match as first argument. In this NodeJS script it will be available as process.argv[2]. This subsitiation-handler needs to write the new match string to stdout so comby can pick up the value and replace it.

// custom.js
#!/usr/bin/env node

console.log(process.argv[2].toUpperCase())

Does this sound sensible to you?

[rvantonder]

That does make sense. GNU sed has something like this (-e).

I think this is a good idea. Adding the process fork/communication for like this will need some effort. I'm going to keep this in mind (and leave the issue open) until I have a better idea of when I have time for this one.

[elfring]

I am also curious how this software will evolve further in mentioned areas.