These notes are for Pig 0.18.0 release. Highlights ========== The highlights of this release are - Support for Hadoop 3, Tez 0.10, Spark 3, Hive 3 and HBase 2 and making them the default - Support for Python 3 in streaming System Requirements =================== 1. Java 1.8.x or newer. Set JAVA_HOME to the root of your Java installation 2. Ant build tool: http://ant.apache.org - to build source only 3. This release provides binaries compatible with Hadoop 2.10.x and 3.x releases. - pig-0.18.0.tar.gz binary release is compatible with Hadoop 3, Spark 3, Hive 3 and HBase 2. - pig-0.18.0-hadoop2.tar.gz binary release is compatible with Hadoop 2, Spark 2, Hive 1 and HBase 1. Trying the Release ================== 1. Download pig-0.18.0.tar.gz 2. Unpack the file: tar -xzvf pig-0.18.0.tar.gz 3. Move into the installation directory: cd pig-0.18.0 4. To run pig without Hadoop cluster, execute the command below. This will take you into an interactive shell called grunt that allows you to navigate the local file system and execute Pig commands against the local files bin/pig -x local 5. To run on your Hadoop cluster, you need to have HADOOP_HOME/HADOOP_PREFIX and HADOOP_CONF_DIR environment variables to point to your hadoop installation and then run pig. To run with Tez or Spark as execution engine instead of MapReduce, install them in your Hadoop cluster. The commands below will take you into an interactive shell called grunt that allows you to navigate Hadoop DFS and execute Pig commands against it. To execute in Tez or Spark mode, run with -x tez or -x spark option. bin/pig 6. To build your own version of pig.jar run ant 7. To run unit tests run ant test 8. To build jar file with available user defined functions run commands below. cd contrib/piggybank/java ant 9. To build the tutorial: cd tutorial ant 10. To run tutorial follow instructions in http://wiki.apache.org/pig/PigTutorial Relevant Documentation ====================== Pig Documentation: http://pig.apache.org/docs/r0.17.0/ Pig Wiki: https://cwiki.apache.org/confluence/display/PIG/Index Pig Tutorial: https://cwiki.apache.org/confluence/display/PIG/PigTutorial