hive (version 0.2-2)

configuration: Managing the Hadoop configuration

Description

Functions for showing/changing Hadoop configuration.

Usage

hive_get_parameter( x, henv = hive() )
hive_get_masters( henv = hive() )
hive_get_workers( henv = hive() )
hive_get_nreducer( henv = hive() )
hive_set_nreducer( n, henv = hive() )

Arguments

henv

An object containing the local Hadoop configuration.

x

A character string naming the parameter in the Hadoop configuration.

n

An integer specifying the number of reducers to be used in hive_stream().

Value

hive_get_parameter() returns the specified parameter as a character string.

hive_get_workers() returns a character vector naming the hostnames of the configured worker nodes in the cluster.

hive_get_masters() returns a character vector of the hostnames of the configured master nodes in the cluster.

hive_get_nreducer() returns an integer representing the number of configured reducers.

Details

The function hive_get_parameter() is used to get parameters from the Hadoop cluster configuration.

The functions hive_get_workers() and hive_get_masters() return the hostnames of the configured nodes in the cluster.

The functions hive_get_nreducer() and hive_set_nreducer() are used to get/set the number of reducers which are used in Hadoop Streaming using hive_stream().

References

Apache Hadoop cluster configuration (https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Configuring_Hadoop_in_Non-Secure_Mode).

Examples

# NOT RUN {
## Which tmp directory is set in the Hadoop configuration?
# }
# NOT RUN {
hive_get_parameter("hadoop.tmp.dir")
# }
# NOT RUN {
## The master nodes of the cluster
# }
# NOT RUN {
hive_get_masters()
# }
# NOT RUN {
## The worker nodes of the cluster
# }
# NOT RUN {
hive_get_workers()
# }
# NOT RUN {
## The number of configured reducers
# }
# NOT RUN {
hive_get_nreducer()
# }
# NOT RUN {
# }