RichyHBM

Software engineer with a focus on game development and scalable backend development

Building a Standalone Scala Executable

A while back whilst researching languages for a new set of tools, it became apparent that ease of deployment was a definite benefit. On one hand there was Go which can be compiled to a single native binary, on the other there was Scala a language I personally preferred due to its compatibility with Java libraries and tools, as well as it forcing me to learn functional programming, but that has a lot of dependencies in the form of the JVM.

Recently I heard about a project called Avian, which is a lightweight alternative to the JVM designed for smaller systems or for embedding into other programs. This got me thinking whether it would be possible to have a Scala application in a single executable without any external dependencies.

As it turned out, it was. Avian has some documentation for embedding Java programs into an executable which for the most part cover the steps required to get Scala embedded, however there were some additional steps that I couldn’t see documented anywhere and therefore am documenting the whole process here.

To begin I should mention these steps were done on a Mac, and so if you are using a different platform the commands may need some small alterations.

Firstly I define some system variables, these are mostly to do with the build environment and location of files. It should all be fairly straightforward aside from the TAILS variable, this essentially dictates whether Avian should be compiled with proper tail recursion:

tails — if true, optimize each tail call by replacing the caller’s stack frame with the callee’s. This convention ensures proper tail recursion, suitable for languages such as Scheme.

PLATFORM=macosx
ARCH=x86_64
TAILS=true
SCALA_VERSION=2.11.8
TOP_LEVEL_DIR=$(pwd)
AVIAN_BUILD_DIR=avian/build/$PLATFORM-$ARCH

if [ $TAILS ]; then
  AVIAN_BUILD_DIR=avian/build/$PLATFORM-$ARCH-tails
fi

Next I check if Avian has been checked out, and do so if not. Here I check out a specific commit as I know this one works, but feel free to check out the latest commit.

if [ ! -d “avian” ]; then
  git clone https://github.com/ReadyTalk/avian.git
  git reset — hard edbce08
fi

Once you have the Avian sources it’s time to compile Avian, there are plenty of compilation options so check out which ones you may want to change here, I just took a simple approach.

export JAVA_HOME=$(/usr/libexec/java_home)
cd $TOP_LEVEL_DIR/avian
make platform=$PLATFORM tails=$TAILS

With Avian now compiled, I created a folder to hold all of the intermediary build files. I then downloaded the Scala library and extracted its contents removing some unneeded files.

cd $TOP_LEVEL_DIR
mkdir build
cd build

if [ ! -f scala-library.jar ]; then
 wget -O scala-library.jar http://central.maven.org/maven2/org/scala-lang/scala-library/$SCALA_VERSION/scala-library-$SCALA_VERSION.jar
fi

unzip scala-library.jar
rm -rf META-INF
rm rootdoc.txt

Next up I extracted the libavian libraries into the build directory, and copied over the Avian JVM classpath.

ar x $TOP_LEVEL_DIR/$AVIAN_BUILD_DIR/libavian.a
cp $TOP_LEVEL_DIR/$AVIAN_BUILD_DIR/classpath.jar boot.jar

Now it’s time to actually compile your Scala code. In this case it is assumed that all Scala code is inside a src folder alongside this bash script, and that the entry to your program is a main method contained in a Main object.

I created a folder to hold all compiled files, and built up a list of Scala files to compile, these were then compiled into said folder specifying the Avian classpath as the classpath to compile against.

I then injected all the compiled .class files into the Avian classpath, along with the Scala library files.

mkdir classes

find $TOP_LEVEL_DIR/src/ -name “*.scala” > sources.txt

scalac -bootclasspath boot.jar -d $TOP_LEVEL_DIR/build/classes @sources.txt
mv boot.jar ./classes/
cd classes
jar u0f boot.jar $(find ./ -name “*.class”)
cd ..
mv ./classes/boot.jar ./
jar u0f boot.jar library.properties scala/

By this point, all your Scala files as well as dependencies should be included in the single Avian classpath jar, so I used the Avian binaryToObject tool to generate a linkable object file from the jar.

$TOP_LEVEL_DIR/$AVIAN_BUILD_DIR/binaryToObject/binaryToObject boot.jar boot-jar.o _binary_boot_jar_start _binary_boot_jar_end $PLATFORM $ARCH

Finally I took the sample C++ program from the Avian build page and compiled it with the Avian classpath object file, and all other Avian libraries. The C++ program simply starts up a JVM finds the Main class, and then the static method main within it, and calls it with the given arguments. If you are not running this on a Mac, you will most likely need to look up what changes are needed to compile this for your platform on the Avian building page.

cp $TOP_LEVEL_DIR/embedded-jar-main.cpp ./

g++ -I$JAVA_HOME/include -I$JAVA_HOME/include/darwin -D_JNI_IMPLEMENTATION_ -c embedded-jar-main.cpp -o main.o

g++ -rdynamic *.o -ldl -lpthread -lz -o scala-avian -framework CoreFoundation

strip -S -x scala-avian

And thats it, you now have a single executable that is capable of starting up a JVM and running some Scala, in my specific test I just used a simple HelloWorld program that makes use of imports to grab data from a different package. I would like to try out some more fully fledged applications in the future, but that may require compiling Avian with the full OpenJDK sources as I have seen some programs fail to run with the standard Avian ones.

For the full files used in this article, see here.

Share!

Follow me!

Subscribe!