Command Line Interface

Hydra can function as a drop-in replacement for the scalac command. This can be useful when using a build tool that doesn't integrate with Hydra yet.

In case your build tool is not directly supported by Hydra, you can still evaluate it by using the stand-alone distribution. Hydra works as a drop-in replacement for the Scala compiler and takes the same command line arguments (and a few extra, like -cpus). If you can extract the exact compiler arguments that are passed to the Scala compiler by your build tool you can run the same command via Hydra.

Note

When running Hydra stand-alone you are using a new (cold) JVM every time you compile. sbt and IntelliJ keep the JVM alive between compilations in order to get it into a warm state. Differences in performance between cold and warm JVM compilation can reach 2x.

Tip

Note: The rest of this site is a good reference, even though it assumes sbt. The Tuning and Troubleshooting guides are very relevant.

Download

You can download a zip distribution of Hydra here. Make sure to choose the distribution matching the exact Scala version that you want to use. This is important when using compiler plugins or macro libraries, since they may depend on compiler internals and are cross-compiled to the full Scala version.

Basics

The runner script is inside the bin/ directory. You can run hydra -help to get an idea of what options are available for the runner script.

You can pass options to the underlying JVM via -J, for instance -J-Xmx4G to pass 4G of heap memory to the Java Virtual Machine. Compiler arguments are separated by --.

A useful option is -Dhydra.logLevel=debug, which enables more detailed logging.

Hydra compiler options

Hydra needs a few additional command line parameters in order to function optimally. Here’s a list of these options, including an example argument


Argument Description
-cpus 4 The number of CPUs that it can use (we recommend using the number of physical cores on your machine, but feel free to experiment -- 4 is a good default)
-sourcepath path1:path2 The equivalent of the classpath. Write down all the paths where your build tool finds sources. Usually this is src/main/scala or src/test/scala, depending what configuration you are going to compile. However, you might have generated sources that live in another directory, for instance sbt places such files in target/scala-2.11/src_managed/main
-YtimingsFile .hydra/timings.csv A file where Hydra can write how long it took to compile, the number of source files, the number of workers, etc.
-YhydraTag project/test A tag to be used as an ID in the timings file. Usually it’s the project name followed by the configuration (main or test)
-YhydraStore .hydra/sharedJS/compile A directory where Hydra can store additional information needed between runs, for instance compilation times for each individual file in the previous full build
-YrootDirectory /Users/dragos/workspace/proj1 The absolute path to the project base directory. In case you are using absolute files for your sources, this will be used to trim them before writing them out (in partition file or timings files)
-YpartitionFile .hydra/subproj1/src/main/partition.hydra A file where Hydra can store an optimal partition of files between workers, based on measured execution times for past executions. This isn’t needed unless using the explicit source partitioner
-YsourcePartitioner:auto A specific strategy for splitting files across workers. By default it’s “auto”, and usually this works best. Another strategy is “explicit”, where the user specifies what files should be assigned to each worker. Usually the default works best.

Argument files

As soon as a project is no longer a toy project the arguments become too long to be passed on the command line each time. Both Hydra and Scala allow reading arguments from a file. The command line becomes:

bin/hydra -J-Xmx4G -- -d /tmp @project.args

Hydra will pick up all arguments found inside the file project.args (you can still add arguments on the command line, like the destination directory above)

For example, a simple arguments file could look like this:

# Hydra arguments
-cpus 4
-sourcepath /Users/dragos/workspace/triplequote/proj1/src/main/scala-2.11

# Regular Scala arguments
-feature
-target:jvm-1.8
-classpath /Users/dragos/workspace/triplequote/proj1/target/scala-2.11/classes:<user.home>/.ivy2/cache/org.scala-js/scalajs-library_2.11/jars/scalajs-library_2.11-0.6.16.jar:....
-unchecked
-deprecation -Xlint:adapted-args -Xlint:nullary-unit -Xlint:inaccessible -Xlint:nullary-override -Xlint:infer-any
# source files follow
src/main/scala/Foo.scala
src/main/scala/Bar.scala

Note that you can add all source files in there. You can use either absolute or relative paths, and a leading # character causes the line to be ignored.

We recommend you add the Hydra-specific parameters at the top of the file.

Depending on the build system, the steps to collect the compiler arguments are slightly different. Read below on how to extract the command line options for Gradle and Maven.

Gradle builds

This section describes how to obtain an args file starting with a Gradle build.

  • Clean output directories (gradle clean).
  • Run gradle with debug options (gradle -d compileScala). Look for the following line in the output:
16:33:37.811 [DEBUG] [org.gradle.api.internal.tasks.scala.ZincScalaCompiler] Calling Scala compiler with arguments  (CompilerInterface):
        -deprecation
        -unchecked
...

Copy everything after this line and paste it in your file.

You should find now all the source files that are built on a clean build. You can do that using command line tools or by looking at the same log messages. Gradle logs source files being compiled at debug mode just like compiler arguments, and usually this comes before them (if it’s a clean build, you should see all your sources logged at a line similar to the following):

16:33:39.972 [DEBUG] [org.gradle.api.internal.tasks.scala.ZincScalaCompiler] [naha] Initial directly invalidated sources: Set(/Users/dragos/sandbox/gradle-test/gradle-scala/src/main/scala/Main.scala, ..)

Copy these files (one per line) in the args file.

Add Hydra-specific arguments to the top of the file and skip to the Benchmarking section.

Maven builds

Maven builds may or may not use the incremental compiler (called Zinc), controlled with the configuration option recompileMode:

<configuration>
  <recompileMode>incremental</recompileMode>
  <useZincServer>true</useZincServer>
</configuration>

No incremental compiler

By default, Maven doesn’t use the incremental compiler. In that case you can obtain the compiler arguments by running Maven with -DdisplayCmd=true:

$ mvn compile -DdisplayCmd=true[INFO] Compiling 561 source files to /Users/dragos/workspace/oss/spark/core/target/scala-2.11/classes at 1502445596279
[INFO] cmd:  /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/bin/java ... scala_maven_executions.MainWithArgsInFile scala.tools.nsc.Main /private/var/folders/1j/_z_4dt6160q58f31nm3xv5c40000gn/T/scala-maven-4881719476242424986.args

The last argument is an args file. Copy and rename it and you just need to add Hydra arguments to the top in order to be able to run Hydra.

Incremental builds

The simplest way is to disable incremental compilation and use the previous section. If that is not possible, run mvn -X to enable debug output and grep the output for a line like the one below:

[DEBUG] Calling Scala compiler with arguments  (CompilerInterface):
    -unchecked
    -deprecation
    -feature
    -explaintypes
    -Yno-adapted-args
    -bootclasspath
...

This will give you all compiler arguments, except for source files. You can find source files a bit earlier in the debug output, after a line like:

[DEBUG] All initially invalidated sources: Set(/Users/dragos/workspace/oss/spark/core/src/main/scala/org/apache/spark/shuffle/BlockStoreShuffleReader.scala, /Users/dragos/workspace/oss/spark/core/src/main/scala/org/apache/spark/status/api/v1/VersionResource.scala, /Users/dragos/workspace/oss/spark/core/src/main/scala/org/apache/spark/util/ThreadStackTrace.scala, ...

Add Hydra-specific arguments to the top of the file and then head over to the Benchmarking section.

Benchmark

Validate the build file

Before building with Hydra, check that the argument file works with vanilla Scala without any Hydra specific options. You may need to download a Scala distribution from http://scala-lang.org/download/, then:

  • Prepare an arguments file according to the instructions above
  • If you added any Hydra options, comment them out with a leading # (-cpus or anything starting with -Yhydra)
  • Run scalac -d /tmp @project.args

If you see any compiler errors (missing types or the like) make sure you haven’t missed any line from the classpath or sources. You may need to give more memory to the Scala compiler if you have a large project. Vanilla Scala doesn’t support -J-Xmx, but you can use the JAVA_OPTS environment variable. On Unix-like systems you can prepend it to the command line

$ JAVA_OPTS=-Xmx4G scalac @project.args

On Windows you may need to use setx and restart the command prompt.

Once this build succeeds you are ready for benchmarking. This will be your baseline time. Benchmarking a JVM process is tricky and there are books written about it. At the minimum:

  • Make sure your computer isn’t doing CPU or memory intensive tasks at the same time (quit all other programs).
  • Disable any backup software, as it might pick up new classfiles and try to back them.
  • Run the Scala compiler several times to account for variability. We recommend at least 5.
  • Delete the output directory contents

Run Hydra

Now that the argument file is valid, add Hydra-specific arguments and run with the Hydra runner. See appendix for an example containing Hydra-specific arguments file.

$ /path/to/hydra/bin/hydra -J-Xmx4g -- @project.args

Make sure that you have all source files, and that -sourcepath is correct. This is especially important for Hydra in order to find dependencies between workers, and missing this argument may lead to “not found” errors

Note that Hydra may print something like

An automatically balanced partition file has been created in <project-dir>/.hydra/myProject/compile/partition.hydra. It will be used by subsequent builds.

Make sure to have an empty output directory every time you run Hydra. This signals Hydra that it’s doing a full build (as opposed to incremental) and it will attempt to optimize the workload between workers for the next run.

You can find more information about this in our Tuning Guide

Benchmarking a JVM process is tricky and there are books written about it. At the minimum:

  • Make sure your computer isn’t doing CPU or memory intensive tasks at the same time (quit all other programs).
  • Disable any backup software, as it might pick up new classfiles and try to back them.
  • Run the Scala compiler several times to account for variability. We recommend at least 5.
  • Delete the output directory contents

For meaningful comparisons your baseline should be vanilla Scalac stand alone (using the same arguments file). Make sure your computer isn’t doing CPU or memory intensive tasks at the same time (quit all other programs).

Appendix

The following args file can be used to compile the common subproject of Specs2 using command line Hydra:

# Hydra arguments
-cpus 4
-sourcepath .../common/src/main/scala/:.../common/target/scala-2.12/:.../common/src/main/scala-2.12.0-RCx/
-YtimingsFile .hydra/timings.csv
-YhydraTag common/compile
-YhydraStore  .hydra/common/compile
-YrootDirectory .../common/

# Vanilla arguments
-Ypartial-unification
-Xplugin:<user.home>/.ivy2/cache/org.spire-math/kind-projector_2.12/jars/kind-projector_2.12-0.9.3.jar
-Xfatal-warnings
-Xlint
-Ywarn-unused-import
-Yno-adapted-args
-Ywarn-numeric-widen
-Ywarn-value-discard
-deprecation:false
-Xcheckinit
-unchecked
-feature
-language:_
-bootclasspath
/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/sunrsasign.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/classes:<user.home>/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.2.jar
-classpath
.../common/target/scala-2.12/classes:<user.home>/.ivy2/cache/org.scalaz/scalaz-effect_2.12/bundles/scalaz-effect_2.12-7.2.7.jar:<user.home>/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.12.2.jar:<user.home>/.ivy2/cache/org.scala-lang.modules/scala-parser-combinators_2.12/bundles/scala-parser-combinators_2.12-1.0.4.jar:<user.home>/.ivy2/cache/org.scala-lang.modules/scala-xml_2.12/bundles/scala-xml_2.12-1.0.5.jar:<user.home>/.ivy2/cache/org.scalaz/scalaz-core_2.12/bundles/scalaz-core_2.12-7.2.7.jar

.../common/src/main/scala/org/specs2/control/NumberOfTimes.scala
.../common/src/main/scala/org/specs2/control/eff/IntoPoly.scala
.../common/src/main/scala/org/specs2/control/ActionException.scala
.../common/src/main/scala/org/specs2/control/eff/StateEffect.scala
.../common/src/main/scala/org/specs2/control/eff/syntax/eff.scala
...