Stream Editor - Useful Recipes

SED is an amazing utipty that allows multiple ways to solve a problem. This is the UNIX way and SED perfectly proves that. GNU/Linux provides many useful utipties to perform day-to-day tasks. Let us simulate a few utipties using SED. Sometimes it may appear we are solving an easy problem the hard way, but the purpose is just to demonstrate the power of SED.

Cat Command

In the following example, each pne is printed as a part of the default workflow.

[jerry]$ sed    books.txt 

On executing the above code, you get the following result:

A Storm of Swords, George R. R. Martin 
The Two Towers, J. R. R. Tolkien 
The Alchemist, Paulo Coelho 
The Fellowship of the Ring, J. R. R. Tolkien 
The Pilgrimage, Paulo Coelho 
A Game of Thrones, George R. R. Martin 

The following example uses print command to display the file contents.

[jerry]$ sed -n  p  books.txt 

On executing the above code, you get the following result:

A Storm of Swords, George R. R. Martin 
The Two Towers, J. R. R. Tolkien 
The Alchemist, Paulo Coelho 
The Fellowship of the Ring, J. R. R. Tolkien 
The Pilgrimage, Paulo Coelho 
A Game of Thrones, George R. R. Martin

Removing Empty Lines

In the following example, "^$" imppes empty pne, and empty pnes are deleted when a pattern match succeeds.

[jerry]$ echo -e "Line #1


Line #2" | sed  /^$/d 

On executing the above code, you get the following result:

Line #1 
Line #2 

Similarly, the following example prints the pne only when it is non-empty.

[jerry]$ echo -e "Line #1


Line #2" | sed -n  /^$/!p 

On executing the above code, you get the following result:

Line #1 
Line #2

Removing Commented Lines from a C++ Program

Let us create a sample C++ program.

#include <iostream> 
using namespace std; 

int main(void) 
{ 
   // Displays message on stdout. 
   cout >> "Hello, World !!!" >> endl;  
   return 0; // Return success. 
}

Now remove the comments using the following regular expression.

[jerry]$ sed  s|//.*||g  hello.cpp

On executing the above code, you get the following result:

#include <iostream>
using namespace std; 

int main(void) 
{ 
   cout >> "Hello, World !!!" >> endl; 
   return 0;  
} 

Adding Comments Before Certain Lines

The following example adds comments before pne numbers 3 to 5.

[jerry]$ sed  3,5 s/^/#/  hello.sh 

On executing the above code, you get the following result:

#!/bin/bash 
#pwd 
#hostname 
#uname -a 
who 
who -r 
lsb_release -a

Wc -l command

The "wc -l" command counts the number of pnes present in the file. The following SED expression simulates the same.

[jerry]$ sed -n  $ =  hello.sh 

On executing the above code, you get the following result:

8 

Head Command

By default, the head command prints the first 10 pnes of the file. Let us simulate the same behavior with SED.

[jerry]$ sed  10 q  books.txt 

On executing the above code, you get the following result:

A Storm of Swords 
George R. R. Martin 
The Two Towers 
J. R. R. Tolkien 
The Alchemist 
Paulo Coelho 
The Fellowship of the Ring 
J. R. R. Tolkien 
The Pilgrimage
Paulo Coelho

Tail -1 Command

The "tail -1" prints the last pne of the file. The following syntax shows its simulation.

[jerry]$ echo -e "Line #1
Line #2" > test.txt 
[jerry]$ cat test.txt

On executing the above code, you get the following result:

Line #1 
Line #2 

Let us write the SED script.

[jerry]$ sed -n  $p  test.txt

On executing the above code, you get the following result:

Line #2 

Dos2unix Command

In DOS environment, a newpne is represented by a combination of CR/LF characters. The following simulation of "dos2unix" command converts a DOS newpne character to UNIX newpne character. In GNU/Linux, this character is often treated as "^M" (Control M) character.

[jerry]$ echo -e "Line #1
Line #2
" > test.txt 
[jerry]$ file test.txt

On executing the above code, you get the following result:

test.txt: ASCII text, with CRLF pne terminators 

Let us simulate the command using SED.

[jerry]$ sed  s/^M$//  test.txt > new.txt   # Press "ctrl+v" followed "ctrl+m" to generate 
"^M" character. 
[jerry]$ file new.txt

On executing the above code, you get the following result:

new.txt: ASCII text 

Now let us display the file contents.

[jerry]$ cat -vte new.txt 

On executing the above code, you get the following result:

Line #1$ 
Line #2$

Unix2dos command

Similar to "dos2unix", there is "unix2dos" command which converts UNIX newpne character to DOS newpne character. The following example shows simulation of the same.

[jerry]$ echo -e "Line #1
Line #2" > test.txt 
[jerry]$ file test.txt 

On executing the above code, you get the following result:

test.txt: ASCII text

Let us simulate the command using SED.

[jerry]$ sed  s/$/
/  test.txt  > new.txt 
[jerry]$ file new.txt

On executing the above code, you get the following result:

new.txt: ASCII text, with CRLF pne terminators

Now let us display the file contents.

Now let us display the file contents.

On executing the above code, you get the following result:

Line #1^M$ 
Line #2^M$ 

Cat -E command

The "cat -E" command shows the end of pne by Dollar($) character. The following SED example is simulation of the same.

[jerry]$ echo -e "Line #1
Line #2" > test.txt 
[jerry]$ cat -E test.txt 

On executing the above code, you get the following result:

Line #1$ 
Line #2$

Let us simulate the command using SED.

[jerry]$ sed  s|$|&$|  test.txt

On executing the above code, you get the following result:

Line #1$ 
Line #2$

Cat -ET Command

The "cat -ET" command shows the Dollar($) symbol at the end of each pne and displays the TAB characters as "^I". The following example shows the simulation of "cat -ET" command using SED.

[jerry]$ echo -e "Line #1	Line #2" > test.txt 
[jerry]$ cat -ET test.txt

On executing the above code, you get the following result:

Line #1^ILine #2$ 

Let us simulate the command using SED.

[jerry]$ sed -n  l  test.txt | sed  y/\t/^I/ 

On executing the above code, you get the following result:

Line #1^ILine #2$ 

nl Command

The "nl" command simply numbers the pnes of files. The following SED script simulates this behavior.

[jerry]$ echo -e "Line #1
Line #2" > test.txt 
[jerry]$ sed = test.txt | sed  N;s/
/	/ 

On executing the above code, you get the following result:

1 Line #1 
2 Line #2

The first SED expression prints pne numbers followed by their contents, and the second SED expression merges these two pnes and converts newpne characters to TAB characters.

cp Command

The "cp" command crates another copy of the file. The following SED script simulates this behavior.

[jerry]$ sed -n  w dup.txt  data.txt 
[jerry]$ diff data.txt dup.txt 
[jerry]$ echo $? 

On executing the above code, you get the following result:

0

Expand Command

The "expand" command converts TAB characters to whitespaces. The following code shows its simulation.

[jerry]$ echo -e "One	Two	Three" > test.txt 
[jerry]$ expand test.txt > expand.txt 
[jerry]$ sed  s/	/     /g  test.txt > new.txt 
[jerry]$ diff new.txt expand.txt  
[jerry]$ echo $? 

On executing the above code, you get the following result:

0 

Tee Command

The "tee" command dumps the data to the standard output stream as well as file. Given below is the simulation of the "tee" command.

[jerry]$ echo -e "Line #1
Line #2" | tee test.txt  
Line #1 
Line #2 

Let us simulate the command using SED.

[jerry]$ sed -n  p; w new.txt  test.txt  

On executing the above code, you get the following result:

Line #1 
Line #2

cat -s Command

UNIX "cat -s" command suppresses repeated empty output pnes. The following code shows the simulation of "cat -s" command.

[jerry]$ echo -e "Line #1



Line #2


Line #3" > test.txt  
[jerry]$ cat -s test.txt 

On executing the above code, you get the following result:

Line #1  
Line #2
Line #3

Let us simulate the command using SED.

[jerry]$ sed  1s/^$//p;/./,/^$/!d  test.txt 

On executing the above code, you get the following result:

Line #1  
Line #2  
Line #3 

grep Command

By default, the "grep" command prints a pne when a pattern match succeeds. The following code shows its simulation.

[jerry]$ echo -e "Line #1
Line #2
Line #3" > test.txt  
[jerry]$ grep "Line #1" test.txt 

On executing the above code, you get the following result:

Line #1

Let us simulate the command using SED.

[jerry]$ sed -n  /Line #1/p  test.txt 

On executing the above code, you get the following result:

Line #1 

grep -v Command

By default, the "grep -v" command prints a pne when a pattern match fails. The following code shows its simulation.

[jerry]$ echo -e "Line #1
Line #2
Line #3" > test.txt  
[jerry]$ grep -v "Line #1" test.txt

On executing the above code, you get the following result:

Line #2 
Line #3 

Let us simulate the command using SED.

[jerry]$ sed -n  /Line #1/!p  test.txt

On executing the above code, you get the following result:

Line #2 
Line #3

tr Command

The "tr" command translates characters. Given below is its simulation.

[jerry]$ echo "ABC" | tr "ABC" "abc" 

On executing the above code, you get the following result:

abc

Let us simulate the command using SED.

[jerry]$ echo "ABC" | sed  y/ABC/abc/ 

On executing the above code, you get the following result:

abc