Apache Pig - Grunt Shell

After invoking the Grunt shell, you can run your Pig scripts in the shell. In addition to that, there are certain useful shell and utipty commands provided by the Grunt shell. This chapter explains the shell and utipty commands provided by the Grunt shell.

Note − In some portions of this chapter, the commands pke Load and Store are used. Refer the respective chapters to get in-detail information on them.

Shell Commands

The Grunt shell of Apache Pig is mainly used to write Pig Latin scripts. Prior to that, we can invoke any shell commands using sh and fs.

sh Command

Using sh command, we can invoke any shell commands from the Grunt shell. Using sh command from the Grunt shell, we cannot execute the commands that are a part of the shell environment (ex − cd).

Syntax

Given below is the syntax of sh command.

grunt> sh shell command parameters

Example

We can invoke the ls command of Linux shell from the Grunt shell using the sh option as shown below. In this example, it psts out the files in the /pig/bin/ directory.

grunt> sh ls
   
pig 
pig_1444799121955.log 
pig.cmd 
pig.py

fs Command

Using the fs command, we can invoke any FsShell commands from the Grunt shell.

Syntax

Given below is the syntax of fs command.

grunt> sh File System command parameters

Example

We can invoke the ls command of HDFS from the Grunt shell using fs command. In the following example, it psts the files in the HDFS root directory.

grunt> fs –ls
  
Found 3 items
drwxrwxrwx   - Hadoop supergroup          0 2015-09-08 14:13 Hbase
drwxr-xr-x   - Hadoop supergroup          0 2015-09-09 14:52 seqgen_data
drwxr-xr-x   - Hadoop supergroup          0 2015-09-08 11:30 twitter_data

In the same way, we can invoke all the other file system shell commands from the Grunt shell using the fs command.

Utipty Commands

The Grunt shell provides a set of utipty commands. These include utipty commands such as clear, help, history, quit, and set; and commands such as exec, kill, and run to control Pig from the Grunt shell. Given below is the description of the utipty commands provided by the Grunt shell.

clear Command

The clear command is used to clear the screen of the Grunt shell.

Syntax

You can clear the screen of the grunt shell using the clear command as shown below.

grunt> clear

help Command

The help command gives you a pst of Pig commands or Pig properties.

Usage

You can get a pst of Pig commands using the help command as shown below.

grunt> help

Commands: <pig latin statement>; - See the PigLatin manual for details:
http://hadoop.apache.org/pig
  
File system commands:fs <fs arguments> - Equivalent to Hadoop dfs  command:
http://hadoop.apache.org/common/docs/current/hdfs_shell.html
	 
Diagnostic Commands:describe <apas>[::<apas] - Show the schema for the apas.
Inner apases can be described as A::B.
    explain [-script <pigscript>] [-out <path>] [-brief] [-dot|-xml] 
       [-param <param_name>=<pCram_value>]
       [-param_file <file_name>] [<apas>] - 
       Show the execution plan to compute the apas or for entire script.
       -script - Explain the entire script.
       -out - Store the output into directory rather than print to stdout.
       -brief - Don t expand nested plans (presenting a smaller graph for overview).
       -dot - Generate the output in .dot format. Default is text format.
       -xml - Generate the output in .xml format. Default is text format.
       -param <param_name - See parameter substitution for details.
       -param_file <file_name> - See parameter substitution for details.
       apas - Apas to explain.
       dump <apas> - Compute the apas and writes the results to stdout.

Utipty Commands: exec [-param <param_name>=param_value] [-param_file <file_name>] <script> -
       Execute the script with access to grunt environment including apases.
       -param <param_name - See parameter substitution for details.
       -param_file <file_name> - See parameter substitution for details.
       script - Script to be executed.
    run [-param <param_name>=param_value] [-param_file <file_name>] <script> -
       Execute the script with access to grunt environment.
		 -param <param_name - See parameter substitution for details.         
       -param_file <file_name> - See parameter substitution for details.
       script - Script to be executed.
    sh  <shell command> - Invoke a shell command.
    kill <job_id> - Kill the hadoop job specified by the hadoop job id.
    set <key> <value> - Provide execution parameters to Pig. Keys and values are case sensitive.
       The following keys are supported:
       default_parallel - Script-level reduce parallepsm. Basic input size heuristics used 
       by default.
       debug - Set debug on or off. Default is off.
       job.name - Single-quoted name for jobs. Default is PigLatin:<script name>     
       job.priority - Priority for jobs. Values: very_low, low, normal, high, very_high.
       Default is normal stream.skippath - String that contains the path.
       This is used by streaming any hadoop property.
    help - Display this message.
    history [-n] - Display the pst statements in cache.
       -n Hide pne numbers.
    quit - Quit the grunt shell. 

history Command

This command displays a pst of statements executed / used so far since the Grunt sell is invoked.

Usage

Assume we have executed three statements since opening the Grunt shell.

grunt> customers = LOAD  hdfs://localhost:9000/pig_data/customers.txt  USING PigStorage( , );
 
grunt> orders = LOAD  hdfs://localhost:9000/pig_data/orders.txt  USING PigStorage( , );
 
grunt> student = LOAD  hdfs://localhost:9000/pig_data/student.txt  USING PigStorage( , );
 

Then, using the history command will produce the following output.

grunt> history

customers = LOAD  hdfs://localhost:9000/pig_data/customers.txt  USING PigStorage( , ); 
  
orders = LOAD  hdfs://localhost:9000/pig_data/orders.txt  USING PigStorage( , );
   
student = LOAD  hdfs://localhost:9000/pig_data/student.txt  USING PigStorage( , );
 

set Command

The set command is used to show/assign values to keys used in Pig.

Usage

Using this command, you can set values to the following keys.

quit Command

You can quit from the Grunt shell using this command.

Usage

Quit from the Grunt shell as shown below.

grunt> quit

Let us now take a look at the commands using which you can control Apache Pig from the Grunt shell.

exec Command

Using the exec command, we can execute Pig scripts from the Grunt shell.

Syntax

Given below is the syntax of the utipty command exec.

grunt> exec [–param param_name = param_value] [–param_file file_name] [script]

Example

Let us assume there is a file named student.txt in the /pig_data/ directory of HDFS with the following content.

Student.txt

001,Rajiv,Hyderabad
002,siddarth,Kolkata
003,Rajesh,Delhi

And, assume we have a script file named sample_script.pig in the /pig_data/ directory of HDFS with the following content.

Sample_script.pig

student = LOAD  hdfs://localhost:9000/pig_data/student.txt  USING PigStorage( , ) 
   as (id:int,name:chararray,city:chararray);
  
Dump student;

Now, let us execute the above script from the Grunt shell using the exec command as shown below.

grunt> exec /sample_script.pig

Output

The exec command executes the script in the sample_script.pig. As directed in the script, it loads the student.txt file into Pig and gives you the result of the Dump operator displaying the following content.

(1,Rajiv,Hyderabad)
(2,siddarth,Kolkata)
(3,Rajesh,Delhi) 

kill Command

You can kill a job from the Grunt shell using this command.

Syntax

Given below is the syntax of the kill command.

grunt> kill JobId

Example

Suppose there is a running Pig job having id Id_0055, you can kill it from the Grunt shell using the kill command, as shown below.

grunt> kill Id_0055

run Command

You can run a Pig script from the Grunt shell using the run command

Syntax

Given below is the syntax of the run command.

grunt> run [–param param_name = param_value] [–param_file file_name] script

Example

Let us assume there is a file named student.txt in the /pig_data/ directory of HDFS with the following content.

Student.txt

001,Rajiv,Hyderabad
002,siddarth,Kolkata
003,Rajesh,Delhi

And, assume we have a script file named sample_script.pig in the local filesystem with the following content.

Sample_script.pig

student = LOAD  hdfs://localhost:9000/pig_data/student.txt  USING
   PigStorage( , ) as (id:int,name:chararray,city:chararray);

Now, let us run the above script from the Grunt shell using the run command as shown below.

grunt> run /sample_script.pig

You can see the output of the script using the Dump operator as shown below.

grunt> Dump;

(1,Rajiv,Hyderabad)
(2,siddarth,Kolkata)
(3,Rajesh,Delhi)

Note − The difference between exec and the run command is that if we use run, the statements from the script are available in the command history.