Ru: Ruby in Your Shell
The venerable sed and awk are extremely powerful text-processing tools. In the hands of a master, these power tools can bend text into almost any shape and form. Unfortunately, I’m no master. More importantly, I don’t really want to invest too much time into learning sed and awk (or even bash!), especially when most of my text processing tasks are ad-hoc.
Enter Ru. Ru let’s you, the lazy Ruby programmer, use your favorite programming language in the shell. This means that you can map, sum, and capitalize to your heart’s content. In this article, we will explore some of the nifty things that Ru lets you do, and how it can simplify your text-processing tasks.
Installing Ru
Ru comes packaged as a Ruby gem. To install Ru, fire up your terminal:
% gem install ru
Successfully installed ru-0.1.4
1 gem installedThe Basics
Before we get to the good bits, let’s learn a bit about how Ru lets us use Ruby to interact with the shell. After installing the gem, we can invoke Ru with the ru command. Similar to typical Unix commands, ru reads from standard input (STDIN), as well as files.
Here’s the first example file for today’s exploration:
% cat a_tale.txt
It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair…ru can take a Ruby command and a file name as an argument:
% ru 'map(&:upcase)' a_tale.txt
IT WAS THE BEST OF TIMES, IT WAS THE WORST OF TIMES, IT WAS THE AGE OF WISDOM, IT WAS THE AGE OF FOOLISHNESS, IT WAS THE EPOCH OF BELIEF, IT WAS THE EPOCH OF INCREDULITY, IT WAS THE SEASON OF LIGHT, IT WAS THE SEASON OF DARKNESS, IT WAS THE SPRING OF HOPE, IT WAS THE WINTER OF DESPAIR…The file can also be passed along using pipes:
% cat a_tale.txt | ru 'map(&:upcase)'In fact, it should be of no surprise that you can pipe the output of ru into another command. Suppose I wanted to count the words:
% cat a_tale.txt | ru 'map(&:upcase)' | wc -w
60This is extremely powerful, since ru is just another Unix command that takes an input and manipulates its output.
With the basics out of the way, let’s head on to the fun stuff.
Text-Processing-Fu in Ru
The best way to get a feel of Ru is to see some examples, or better yet, try them out yourself! Here’s the example file that we will work with for the next few examples:
% cat chapters.txt
from zero to deploy
a toy app
mostly static pages
rails-flavored ruby
filling in the layout
modeling users
sign up
log in, log out
updating, showing, and deleting users
account activation and password resets
user microposts
following usersFiltering
Let’s say I want to search for chapter titles that might be too wordy:
% cat chapters.txt | ru 'select { |l| l.split.size == 5 }'
updating, showing, and deleting users
account activation and password resetsInternally, Ru reads the file as an array of lines. The select { ... } is then called on the array. Because it is an array, you can call any Array method.
Skipping Lines
Sometimes, it is useful know about line numbers. In this example, I want to skip odd lines, and remove any empty lines:
% ru 'map.with_index { |line, index| index % 2 == 0 ? line : "" }' chapters.txt | grep -v '^$'
from zero to deploy
mostly static pages
filling in the layout
sign up
updating, showing, and deleting users
user micropostsOnce again, Ru doesn’t stop you from combining the output with existing commands. Here, we are using grep to filter out empty lines.
Titleize and ActiveSupport
Ru comes with ActiveSupport built-in. That is super sweet, because it adds handy functions, such as titleize. This is very useful for the occasional article writer who is absolutely clueless about the rules to titleizing headers:
% ru 'map(&:titleize)' chapters.txt
From Zero To Deploy
A Toy App
Mostly Static Pages
Rails Flavored Ruby
Filling In The Layout
Modeling Users
Sign Up
Log In, Log Out
Updating, Showing, And Deleting Users
Account Activation And Password Resets
User Microposts
Following Users(OK, maybe it doesn’t do it exactly right, but it’s still cool…)
Unix VS Ruby
Say I wanted to find the top 10 commands I frequently use. In a completely Unix-land, I might do:
% history | awk '{print $2;}' | sort | uniq -c | sort -rn | head -10This returns an output like:
1737 cd
1052 git
880 ls
857 vim
608 gst
310 mix
263 exit
248 gc
240 fg
224 iexWhile my head can intuitively understand what is going on, I’m not too sure about all the additional flags needed. Also, I cheated and had to search StackOverflow for the above incantation.
Here is the equivalent in Ru:
history | ru 'inject(Hash.new(0)) { |h, l| h[l.split[1]] += 1; h }.sort_by { |k, v| -v }.map(&:first).take(10)'Ruby is great because of its readable syntax and expressiveness, and the Rubyist in me understands perfectly what is going on, even if it seems slightly more verbose.
JSON Formatter
I’ve always had to prettify JSON at some point during web development. For example, take the following JSON:
% cat sample.json
{ "id": "0001", "type": "donut", "name": "Cake", "ppu": 0.55, "batters": { "batter": [ { "id": "1001", "type": "Regular" }, { "id": "1002", "type": "Chocolate" }, { "id": "1003", "type": "Blueberry" }, { "id": "1004", "type": "Devil's Food" } ] }, "topping": [ { "id": "5001", "type": "None" }, { "id": "5002", "type": "Glazed" }, { "id": "5005", "type": "Sugar" }, { "id": "5007", "type": "Powdered Sugar" }, { "id": "5006", "type": "Chocolate with Sprinkles" }, { "id": "5003", "type": "Chocolate" }, { "id": "5004", "type": "Maple" } ] }Python comes with a nifty tool that does this:
cat sample.json | python -m json.toolI’ve been suffering from some Python envy because of this. Let’s try to achieve a similar effect in Ru:
% cat sample.json | ru 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'This gives you the expected prettified JSON:
% cat sample.json | ru 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'
{
"id": "0001",
"type": "donut",
"name": "Cake",
"ppu": 0.55,
"batters": {
"batter": [
{
"id": "1001",
"type": "Regular"
},
{
"id": "1002",
"type": "Chocolate"
},
{
"id": "1003",
"type": "Blueberry"
},
{
"id": "1004",
"type": "Devil's Food"
}
]
},
"topping": [
{
"id": "5001",
"type": "None"
},
{
"id": "5002",
"type": "Glazed"
},
{
"id": "5005",
"type": "Sugar"
},
{
"id": "5007",
"type": "Powdered Sugar"
},
{
"id": "5006",
"type": "Chocolate with Sprinkles"
},
{
"id": "5003",
"type": "Chocolate"
},
{
"id": "5004",
"type": "Maple"
}
]
}There’s a couple of things to notice here. First, we can do require just like any Ruby script. Take a look how we managed to do the JSON prettification again:
% cat sample.json | ru 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'So far, we have assumed that the first argument is implicitly the file. However, in this case, the lines in the file are captured in the files method. We need to call join in this case because JSON.parse expects a String as input.
Obviously, typing so much just to get JSON prettification is a major hassle. Before you develop any Python envy yourself, Ru has a nice trick up its sleeve – ru save. With ru save , you can name a command and save it. For example, to save the JSON prettifier:
% ru save jsonify 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'
Saved command: jsonify is 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'Then we can run the command with ru run command:
% cat sample.json | ru run jsonifyTo get a list of saved commands, type ru list:
% ru list
Saved commands:
jsonify require "json"; JSON.pretty_generate(JSON.parse(files.join))Ru Helps You Simplify Your Workflow
While I don’t use Ru that often, I’m glad that I always have it in my toolbox. To learn more about Ru, head over to the official page. There are some excellent examples showcased, such as:
Thanks for reading!
