Installing Java

Presto is written in Java and requires a Java JVM to be installed on your system. Presto often requires a recent update of the latest major release. For example, at the time of the writing of this book, Presto requires at least Java version 8 update 151 (Java 8u151) or Java 11. If you do not have that version of Java, Presto will fail to start. Presto typically moved to requires the recent versions of Java in order to take advantage of the new features, but more importantly performance, security, and bug fixes and improvements. You should always refer to the Presto documentation for the latest Java requirements.

Presto will run using Oracle Java and OpenJDK, but not limited to those. You can download Oracle Java from https://www.oracle.com/java/ and OpenJDK from https://openjdk.java.net/.

Installing Presto

We will use the wget and tar commands to download and extract the Presto Server tarball on a Linux System.

$ wget  http://central.maven.org/maven2/io/prestosql/presto-server/305/

            presto-server-305.tar.gz
$ tar xvzf presto-server-305.tar.gz

Once you extract the Presto Server tarball, it will create a single top-level directory, presto-server-305. This directory is referred to as the installation directory. The installation directory contains:

lib/

This directory contains the the JARs (Java archive) that make up the Presto Server.

plugins/

This directory contains the Presto plugin JARs. By default Presto is packaged with many plugins, but third-party plugins may be added as well. We will cover Presto plugins in Chapter XX, but Presto allows for pluggable components to integrate with Presto such as connectors, functions, and security access controls.

bin/

This directory contains helper launcher scripts for Presto. These launcher scripts are use to start, stop, restart, kill and get the status a running Presto process. They also provide options to override default behaviors. We’ll discuss this in great detail in the next section.

etc/

This directory is the configuration directory. It is created by the user and provide the necessary configurations needed by Presto. By default, it’s assumed the configurations are located within the installation directory. But they may be placed elsewhere in the filesystem.

var/

Finally, there is a data directory. This is the place where logs are stored and it is created the first time Presto Server is launched. By default it’s located in the installation directly. However, it is recommended to configure it outside of the installation directory to allow for the data to be preserved across upgrades.

Presto Configuration

Before we can start Presto, we need to provide a set of configuration files.

• Presto Server Configuration

• Presto Catalog Configuration

• Presto Logging Configuration

• Presto Node Configuration

• Java Virtual Machine (JVM) Configuration

It’s very important to point out that these configurations must exist on every machine where Presto runs. Generally, the content in the configurations across a cluster of Presto nodes are the same. We will touch on when they are different a bit later.

By default, the configuration files are expected to be located in an etc directory inside the installation directory. But the defaults can be changed. This will be explained in the section detailing the Presto launcher scripts.

With the exception of the JVM configuration, the above configurations follow the Java Properties standards. As a general description for Java Properties, each configuration parameter is stored as a pair of string in the format key=value. If you’re interested in learning about the full specification of Java Properties, you should refer to the full Java documentation on the topic. There are more advanced features such as trimmings, escapes, line continuations, and encodings. But as a general recommendation, it’s usually best to stay to using the basic key=value feature.. For example, Presto will not interpret the full specification for the node.properties file.

connector.name=tpch

io.prestosql=INFO

node.environment=production
 node.id=ffffffff-ffff-ffff-ffff-ffffffffffff
 node.data-dir=/var/presto/data

-server
 -mx16G
 -XX:+UseG1GC
 -XX:G1HeapRegionSize=32M
 -XX:+UseGCOverheadLimit
 -XX:+ExplicitGCInvokesConcurrent
 -XX:+HeapDumpOnOutOfMemoryError
 -XX:+ExitOnOutOfMemoryError

Using Presto on a Single Machine

The simplest first step to get started with Presto is to run Presto on a single machine. Inside the Presto installation directory we created in a previous section, let’s create the basic set of Presto configurations. We will use vi to create the configuration file. Please use the contents below for the files.

$ cd presto-server-305
 $ mkdir -p etc/catalog
 $ vim etc/config.properties
 $ vim etc/node.properties
 $ vim etc/jvm.config
 $ vim etc/catalog/tpch.properties

For the configuration files using the following content.

etc/config.properties

coordinator=true
 node-scheduler.include-coordinator=true
 http-server.http.port=8080
 query.max-memory=5GB
 query.max-memory-per-node=1GB
 query.max-total-memory-per-node=2GB
 discovery-server.enabled=true
 discovery.uri=http://localhost:8080

etc/node.properties

node.environment=demo
 node.id=ffffffff-ffff-ffff-ffff-ffffffffffff
 node.data-dir=/var/presto/data

etc/jvm.config

-server
 -Xmx4G
 -XX:+UseG1GC
 -XX:G1HeapRegionSize=32M
 -XX:+UseGCOverheadLimit
 -XX:+ExplicitGCInvokesConcurrent
 -XX:+HeapDumpOnOutOfMemoryError
 -XX:+ExitOnOutOfMemoryError
 -Djdk.nio.maxCachedBufferSize=2000000

etc/catalog/tpch.properties

connector.name=tpch

The list of commands and configuration can be found on the GitHub repository we set up for this book. You can find it here: https://www.github.com/{{TODO}}.

Starting Presto

The installation directory contains a couple of launcher scripts. We will use those to start Presto.

$ bin/launcher run

This command will run Presto as a foreground process. Logs and other output to Presto are written to stdout and stderr. When Presto is started you should see this at the bottom of the output.

YYYY-MM-DDTHH:MM:SS INFO main 
 io.prestosql.server.PrestoServer ======== SERVER STARTED ========

Running Presto in the foreground can be useful for quickly verifying whether the process starts up correctly and that it is using the expected configuration settings. In a production environment, you will typically run it as a background daemon process. You can do so via the following command.

$ bin/launcher start
 Started as 48322

The number 48322 you see in the example above if the process ID (pid). The number you’ll see will be different.

Stopping Presto

To stop Presto running as a daemon you should run the following

$ bin/launcher stop
 Stopped 48322

To forcefully stop Presto you can run the following

$ bin/launcher kill
 Killed 48322

Get Status

You can obtain the status of Presto by running:

$ bin/launcher status
 Running as 48322

If Presto is not running you will see this

$ bin/launcher status
 Not running

In the Presto Command Line Interface below we will explain how to connect to a Presto Server. If you wish to follow that exercise, you should keep the Presto Server running here.

Presto Server Logs

After starting Presto as a daemon, you’ll find log files in var/log. This will be located within the installation directly unless you specified a different location in the etc/node.properties file.

launcher.log

This log is created by the launcher and is connected to stdout and stderr streams of the server. It will contain a few log messages that occur while the server logging is being initialized and any errors or diagnostics produced by the JVM.

server.log

This is the main log file used by Presto. It will typically contain the relevant information if the server fails during initialization. It is automatically rotated and compressed.

http-request.log

This is the HTTP request log which contains every HTTP request received by the server. It is automatically rotated and compressed.

Presto Launcher Options

The following command shows the list of options for the Presto Launcher.

$ bin/launcher --help
 Usage: launcher [options] command
 Commands: run, start, stop, restart, kill, status
 Options:
 -h, --help show this help message and exit
 -v, --verbose Run verbosely
 --etc-dir=DIR Defaults to INSTALL_PATH/etc
 --launcher-config=FILE
 Defaults to INSTALL_PATH/bin/launcher.properties
 --node-config=FILE Defaults to ETC_DIR/node.properties
 --jvm-config=FILE Defaults to ETC_DIR/jvm.config
 --config=FILE Defaults to ETC_DIR/config.properties
 --log-levels-file=FILE
 Defaults to ETC_DIR/log.properties
 --data-dir=DIR Defaults to INSTALL_PATH
 --pid-file=FILE Defaults to DATA_DIR/var/run/launcher.pid
 --launcher-log-file=FILE
 Defaults to DATA_DIR/var/log/launcher.log (only in daemon mode)
 --server-log-file=FILE
 Defaults to DATA_DIR/var/log/server.log (only in daemon mode)
 -D NAME=VALUE Set a Java system property

You will notice that the launcher script allows for the configuration properties to be located in different locations other than within the installation etc directory. For example, we will discuss install Presto via RPM in a following section. Using the RPM installation method will locate the configuration directly.

Examples

presto> show catalogs;
 Catalog 
 ---------
 system 
 tpch 
 (2 rows)
 Query 20181103_210445_00032_te3wy, FINISHED, 1 node
 Splits: 19 total, 19 done (100.00%)
 0:00 [0 rows, 0B] [0 rows/s, 0B/s]
 presto> show schemas in tpch like '%3%';
 Schema 
 ---------
 sf300 
 sf3000 
 sf30000 
 (3 rows)
 Query 20181103_210508_00035_te3wy, FINISHED, 1 node
 Splits: 19 total, 19 done (100.00%)
 0:00 [10 rows, 119B] [103 rows/s, 1.2KB/s]
 presto> use tpch.sf300;
 USE
 presto:sf300> show tables;
 Table 
 ----------
 customer 
 lineitem 
 nation 
 orders 
 part 
 partsupp 
 region 
 supplier 
 (8 rows)
 Query 20181103_210604_00037_te3wy, FINISHED, 1 node
 Splits: 19 total, 19 done (100.00%)
 0:00 [8 rows, 174B] [52 rows/s, 1.12KB/s]
 presto:sf300> describe nation;
 Column | Type | Extra | Comment 
 -----------+--------------+-------+---------
 nationkey | bigint | | 
 name | varchar(25) | | 
 regionkey | bigint | | 
 comment | varchar(152) | | 
 (4 rows)
 Query 20181103_210619_00038_te3wy, FINISHED, 1 node
 Splits: 19 total, 19 done (100.00%)
 0:00 [4 rows, 264B] [22 rows/s, 1.43KB/s]

Presto CLI Pager

By default, the results of queries are paginated using the less program which is configured with a carefully selected set of options. This behavior can be overridden by setting the environment variable PRESTO_PAGER to the name of a different program such as more, or set it to an empty value to completely disable pagination.

Presto CLI History

The Presto CLI keeps a history of the previous command run. You can use the up and down arrows to scroll through the history as well as CTRL+S and CTRL+R to search through the history. If you want to execute a query again, press Enter to execute the query.

By default, the Presto history file is located in ~/.presto_history. However, you can change the default by the PRESTO_HISTORY_FILE environment variable.

Additional Diagnostics

The Presto CLI provides the --debug option to enable debug information when running queries.

./presto-cli --debug

presto:sf1> select count(*) from foo;
 Query 20181103_201856_00022_te3wy failed: line 1:22: Table tpch.sf1.foo

            does not exist
 io.prestosql.sql.analyzer.SemanticException: line 1:22: Table tpch.sf1.foo

            does not exist
 at io.prestosql.sql.analyzer.StatementAnalyzer$Visitor.visitTable

            (StatementAnalyzer.java:855)
 at io.prestosql.sql.analyzer.StatementAnalyzer$Visitor.visitTable
 at io.prestosql.presto.execution.SqlQueryManager.createQueryInternal

            (SqlQueryManager.java:390)
 
 ...

 at java.util.concurrent.ThreadPoolExecutor.runWorker

            (ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run

            (ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
          

Output Formats

The Presto CLI provides the command --output-format to control how the output is displayed when running in non-interactive mode. The available options are ALIGNED, VERTICAL, CSV, TSV, CSV_HEADER, TSV_HEADER, NULL. The default value is CSV.

Ignoring Errors

The Presto CLI provides the command --ignore-error if you want to skip any errors that encountered while executing the queries in a file. The default behavior is to stop execution of the script upon encountering the first error.

RPM Installation

Presto can also be installed via RPM Package Manager (RPM). To install Presto using the Presto RPM Package, we will use wget and rpm commands on a Linux system.

$ wget  https://repo1.maven.org/maven2/io/prestosql/presto-server-rpm/305/

            presto-server-rpm-305.rpm
 $ sudo rpm -i presto-server-rpm-305.rpm

This will install Presto in a single node mode and create the basic Presto configuration files and a service control script to control the Presto Server.

service presto [start|stop|restart|status]

catalog.config-dir=/etc/presto/catalog
 plugin.dir=/usr/lib/presto/lib/plugin

$ rpm -e presto