Go to version:

Table Of Contents

  1. Docs
  2. treelite 0.91 documentation

Treelite : model compiler for decision tree ensembles

Treelite is a model compiler for decision tree ensembles, aimed at efficient deployment.

Star Watch

You are currently browsing the documentation of a stable version of treelite: 0.91.

Why treelite?

Compile and optimize your model for fast prediction

Treelite compiles your tree model into optimized shared library. A Benchmark demonstrates 2-6x improvement in prediction throughput, due to more efficient use of compute resources.

Use machine learning package of your choice

Treelite accommodates a wide range of decision tree ensemble models. In particular, it handles both random forests and gradient boosted trees.

Treelite can read models produced by XGBoost, LightGBM, and scikit-learn. In cases where you are using another package to train your model, you may use the flexible builder class.

Deploy with minimal dependencies

It is a great hassle to install machine learning packages (e.g. XGBoost, LightGBM, scikit-learn, etc.) on every machine your tree model will run. This is the case no longer: treelite will export your model as a stand-alone prediction library so that predictions will be made without any machine learning package installed.

Quick start

Install treelite from PyPI:

pip3 install --user treelite

Import your tree ensemble model into treelite:

import treelite
model = treelite.Model.load('my_model.model', model_format='xgboost')

Deploy a source archive:

# Produce a zipped source directory, containing all model information
# Run `make` on the target machine
model.export_srcpkg(platform='unix', toolchain='gcc',
                    pkgpath='./mymodel.zip', libname='mymodel.so',
                    verbose=True)

Deploy a shared library:

# Like export_srcpkg, but generates a shared library immediately
# Use this only when the host and target machines are compatible
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True)

Make predictions on the target machine:

import treelite.runtime
predictor = treelite.runtime.Predictor('./mymodel.so', verbose=True)
batch = treelite.runtime.Batch.from_npy2d(X)
out_pred = predictor.predict(batch)

Read First tutorial for a more detailed example. See Deploying models for additional instructions on deployment.

Note

A note on API compatibility

Since treelite is in early development, its API may change substantially in the future.

Benchmark

See the page Benchmark for details.

How treelite works


(Click to enlarge)

The workflow involves two distinct machines: the host machine that generates prediction subroutine from a given tree model, and the target machine that runs the subroutine. The two machines exchange a single C file that contains all relevant information about the tree model. Only the host machine needs to have treelite installed; the target machine requires only a working C compiler.

Contents

Installation

You may choose one of two methods to install treelite on your system:

Compile treelite from the source

Installation consists of two steps:

  1. Build the shared libraries from C++ code (See the note below for the list.)

  2. Install the Python package.

Note

List of libraries created

There will be two libraries created: the main library, for producing optimized prediction subroutines; and the runtime library, for deploying these subroutines in the wild for actual prediction tasks.

Operating System

Main library

Runtime library

Windows

treelite.dll

treelite_runtime.dll

Mac OS X

libtreelite.dylib

libtreelite_runtime.dylib

Linux / other UNIX

libtreelite.so

libtreelite_runtime.so

To get started, clone treelite repo from GitHub. It is important to clone the submodules with --recursive option.

git clone --recursive https://github.com/dmlc/treelite.git
cd treelite

The next step is to build the shared libraries.

1-1. Compiling shared libraries on Linux and Mac OS X

Here, we use CMake to generate a Makefile:

mkdir build
cd build
cmake ..

Once CMake finished running, simply invoke GNU Make to obtain the shared libraries.

make

The compiled libraries will be under the lib/ directory.

Note

Compiling treelite with multithreading on Mac OS X

The default clang installation on Mac OS X does not support OpenMP, the language construct for multithreading. To enable multithreading in treelite, we recommend that you install gcc 7.x using Homebrew:

brew install gcc@7

After g++ is installed, run CMake again with gcc as the C++ compiler:

cmake .. -DCMAKE_CXX_COMPILER=g++-7 -DCMAKE_C_COMPILER=gcc-7
1-2. Compiling shared libraries on Windows

We can use CMake to generate a Visual Studio project. The following snippet assumes that Visual Studio 2017 is installed. Adjust the version depending on the copy that’s installed on your system.

mkdir build
cd build
cmake .. -G"Visual Studio 15 2017 Win64"

Note

Visual Studio 2015 or newer is required

A large part of treelite has been written using the C++11 standard. Visual Studio 2015 is the first version that supports the new standard to fullest extent.

Once CMake finished running, open the generated solution file (treelite.sln) in Visual Studio. From the top menu, select Build > Build Solution. The compiled libraries will be under the lib/ directory.

2. Installing Python package

The Python package is located at the python subdirectory. There are several ways to install the package:

1. Install system-wide, which requires root permission

cd python
sudo python setup.py install

You will need Python setuptools module for this to work. It is often part of the core Python installation. Should it be necessary, the package can be installed using pip:

pip install -U pip setuptools

2. Install for only current user

This is useful if you do not have the administrative rights.

cd python
python setup.py develop --user

Note

Recompiling treelite

Every time the C++ portion of treelite gets re-compiled, the Python package must be re-installed for the new library to take effect.

3. Set the environment variable PYTHONPATH to locate treelite package

Only set the environment variable PYTHONPATH to tell Python where to find the treelite package. This is useful for developers, as any changes made to C++ code will be immediately visible to Python side without re-running setup.py.

export PYTHONPATH=path/to/treelite/python
python          # enter interactive session

Tutorials

This page lists tutorials about treelite.

First tutorial

This tutorial will demonstrate the basic workflow.

import treelite
Regression Example

In this tutorial, we will use a small regression example to describe the full workflow.

Load the Boston house prices dataset

Let us use the Boston house prices dataset from scikit-learn (sklearn.datasets.load_boston()). It consists of 506 houses with 13 distinct features:

from sklearn.datasets import load_boston
X, y = load_boston(return_X_y=True)
print('dimensions of X = {}'.format(X.shape))
print('dimensions of y = {}'.format(y.shape))
Train a tree ensemble model using XGBoost

The first step is to train a tree ensemble model using XGBoost (dmlc/xgboost).

Disclaimer: Treelite does NOT depend on the XGBoost package in any way. XGBoost was used here only to provide a working example.

import xgboost
dtrain = xgboost.DMatrix(X, label=y)
params = {'max_depth':3, 'eta':1, 'silent':1, 'objective':'reg:linear',
          'eval_metric':'rmse'}
bst = xgboost.train(params, dtrain, 20, [(dtrain, 'train')])
Pass XGBoost model into treelite

Next, we feed the trained model into treelite. If you used XGBoost to train the model, it takes only one line of code:

model = treelite.Model.from_xgboost(bst)

Note

Using other packages to train decision trees

With additional work, you can use models trained with other machine learning packages. See this page for instructions.

Generate shared library

Given a tree ensemble model, treelite will produce a prediction subroutine (internally represented as a C program). To use the subroutine for prediction task, we package it as a dynamic shared library, which exports the prediction subroutine for other programs to use.

Before proceeding, you should decide which of the following compilers is available on your system and set the variable toolchain appropriately:

  • gcc

  • clang

  • msvc (Microsoft Visual C++)

toolchain = 'clang'   # change this value as necessary

The choice of toolchain will be used to compile the prediction subroutine into native code.

Now we are ready to generate the library.

model.export_lib(toolchain=toolchain, libpath='./mymodel.dylib', verbose=True)
                            #                            ^^^^^
                            # set correct file extension here; see the following paragraph

Note

File extension for shared library

Make sure to use the correct file extension for the library, depending on the operating system:

  • Windows: .dll

  • Mac OS X: .dylib

  • Linux / Other UNIX: .so

Note

Want to deploy the model to another machine?

This tutorial assumes that predictions will be made on the same machine that is running treelite. If you’d like to deploy your model to another machine (that may not have treelite installed), see the page Deploying models.

Note

Reducing compilation time for large models

For large models, export_lib() may take a long time to finish. To reduce compilation time, enable the parallel_comp option by writing

model.export_lib(toolchain=toolchain, libpath='./mymodel.dylib',
                 params={'parallel_comp': 32}, verbose=True)

which splits the prediction subroutine into 32 source files that gets compiled in parallel. Adjust this number according to the number of cores on your machine.

Use the shared library to make predictions

Once the shared library has been generated, we feed it into a separate module (treelite.runtime) known as the runtime. The optimized prediction subroutine is exposed through the Predictor class:

import treelite.runtime     # runtime module
predictor = treelite.runtime.Predictor('./mymodel.dylib', verbose=True)

We decide on which of the houses in X we should make predictions for. Say, from 10th house to 20th:

batch = treelite.runtime.Batch.from_npy2d(X, rbegin=10, rend=20)

We used the method from_npy2d() because the matrix X was a dense NumPy array (numpy.ndarray). If X were a sparse matrix (scipy.sparse.csr_matrix), we would have used the method from_csr() instead.

out_pred = predictor.predict(batch)
print(out_pred)

Importing tree ensemble models

Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.

Importing XGBoost models

XGBoost (dmlc/xgboost) is a fast, scalable package for gradient boosting. Both treelite and XGBoost are hosted by the DMLC (Distributed Machine Learning Community) group.

Treelite plays well with XGBoost — if you used XGBoost to train your ensemble model, you need only one line of code to import it. Depending on where your model is located, you should do one of the following:

# bst = an object of type xgboost.Booster
model = Model.from_xgboost(bst)

  • Load XGBoost model from a binary model file

# model had been saved to a file named my_model.model
# notice the second argument model_format='xgboost'
model = Model.load('my_model.model', model_format='xgboost')
Importing LightGBM models

LightGBM (Microsoft/LightGBM) is another well known machine learning package for gradient boosting. To import models generated by LightGBM, use the load() method with argument model_format='lightgbm':

# model had been saved to a file named my_model.txt
# notice the second argument model_format='lightgbm'
model = Model.load('my_model.txt', model_format='lightgbm')
Importing scikit-learn models

Scikit-learn (scikit-learn/scikit-learn) is a Python machine learning package known for its versatility and ease of use. It supports a wide variety of models and algorithms. The following kinds of models can be imported into treelite.

To import scikit-learn models, use treelite.gallery.sklearn.import_model():

# clf is the model object generated by scikit-learn
import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)
How about other packages?

If you used other packages to train your ensemble model, you’d need to specify the model programmatically. There are two ways to do this:

Optimizing prediction subroutine

Treelite offers system-level optimizations to boost prediction performance. Notice that the model information is wholly preserved; the optimizations only affect the manner at which prediction is performed.

Annotate conditional branches

This optimization analyzes and annotates every threshold conditions in the test nodes to improve performance.

How to use

The first step is to generate the branch annotation record for your ensemble model. Make sure to have your training data ready.

# model = your ensemble model (object of type treelite.Model)
# dmat = training data (object of type treelite.DMatrix)

# Create annotator object
annotator = treelite.Annotator()
# Annotate branches by iterating over the training data
annotator.annotate_branch(model=model, dmat=dmat, verbose=True)
# Save the branch annotation record as a JSON file
annotator.save(path='mymodel-annotation.json')

To utilize the branch annotation record, supply the compiler parameter annotate_in when exporting the model:

# Export a source directory
model.compile(dirpath='./mymodel', verbose=True,
              params={'annotate_in': 'mymodel-annotation.json'})

# Export a source directory, packaged in a zip archive
model.export_srcpkg(platform='unix', toolchain='gcc', pkgpath='./mymodel.zip',
                    libname='mymodel.so', verbose=True,
                    params={'annotate_in': 'mymodel-annotation.json'})

# Export a shared library
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True,
                 params={'annotate_in': 'mymodel-annotation.json'})
Technical details
Rationale

Modern CPUs heavily rely on a technique known as branch prediction, in which they “guess” the result of the conditional expression in each if-else branch ahead of time. Given a program

if ( [conditional expression] ) {
  foo();
} else {
  bar();
}

the CPU will pre-fetch the instructions for the function foo() if the given condition is likely to be true. On the other hand, if the condition is likely to be false, the CPU will pre-fetch the instructions for the function bar(). It suffices to say that correctly predicting conditional branches has great impact on performance. Each time the CPU predicts a branch correctly, it can keep the instructions it had pre-fetched earlier. Each time the CPU fails to predict, it must throw away the pre-fetched instructions and fetch anew another set of instructions. If you’d like to learn more about the importance of branch prediction, read this excellent introductory article from Stanford.

The prediction subroutine for a decision tree ensemble is problematic, as it is replete with conditional branches that will have to be guessed well:

/* A slice of prediction subroutine */
float predict_margin(const float* data) {
  float sum = 0.0f;
  if (!(data[0].missing != -1) || data[0].fvalue <= 9.5) {
    if (!(data[0].missing != -1) || data[0].fvalue <= 3.5) {
      if (!(data[10].missing != -1) || data[10].fvalue <= 0.74185) {
        if (!(data[0].missing != -1) || data[0].fvalue <= 1.5) {
          if (!(data[2].missing != -1) || data[2].fvalue <= 2.08671) {
            if ( (data[4].missing != -1) && data[4].fvalue <= 2.02632) {
              if (!(data[3].missing != -1) || data[3].fvalue <= 0.763339) {
                sum += (float)0.00758165;
              } else {
                sum += (float)0.0060202;
              }
            } else {
              if ( (data[1].missing != -1) && data[1].fvalue <= 0.0397456) {
                sum += (float)0.00415399;
              } else {
                sum += (float)0.00821985;
              }
            }
/* and so forth... */

In fact, each threshold condition in the test nodes will need to be predicted. While CPUs lack adequate information to make good guesses on these conditions, we can help by providing that information.

Mechanism for supplying the C compiler with branch information

We predict the likelihood of each condition by counting the number of data points from the training data that satisfy that condition. See the diagram below for an illustration.

If a condition is true at least 50% of the time (over the training data), the condition is labeled as “expected to be true”:

/* expected to be true */
if ( __builtin_expect( [condition], 1 ) ) {
  ...
} else {
  ...
}

On the other hand, if a condition is false at least 50% of the time, the condition is labeled as “expected to be false”:

/* expected to be false */
if ( __builtin_expect( [condition], 0 ) ) {
  ...
} else {
  ...
}

Note

On the expression __builtin_expect

The __builtin_expect expression is a compiler intrinsic to supply the C compiler with branch prediction information. Both gcc and clang support it. Unfortunately, Microsoft Visual C++ does not. To take advantage of branch annotation, make sure to use gcc or clang on the target machine.

Use integer thresholds for conditions

This optimization replaces all thresholds in the test nodes with integers so that each threshold condition performs integer comparison instead of the usual floating-point comparison. The thresholds are said to be quantized into integer indices.

BEFORE:

if (data[3].fvalue < 1.5) {  /* floating-point comparison */
  ...
}

AFTER:

if (data[3].qvalue < 3) {     /* integer comparison */
  ...
}
How to use

Simply add the compiler parameter quantize=1 when exporting the model:

# Export a source directory
model.compile(dirpath='./mymodel', verbose=True,
              params={'quantize': 1})

# Export a source directory, packaged in a zip archive
model.export_srcpkg(platform='unix', toolchain='gcc', pkgpath='./mymodel.zip',
                    libname='mymodel.so', verbose=True,
                    params={'quantize': 1})

# Export a shared library
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True,
                 params={'quantize': 1})
Technical details
Rationale

On some platforms such as x86-64, replacing floating-point thresholds with integers helps improve performance by 1) reducing executable code size and 2) improving data locality. This is so because on these platforms, integer constants can be embedded as part of the comparison instruction, whereas floating-point constants cannot.

Let’s look at x86-64 platform. The integer comparison

a <= 4

produces one assembly instruction:

cmpl    $4, 8(%rsp)       ;    8(%rsp) contains the variable a

Since the integer constant 4 got embedded into the comparison instruction cmpl, we only had to fetch the variable a from memory.

On the other hand, the floating-point comparison

b < 1.2f

produces two assembly instructions:

movss   250(%rip), %xmm0  ;  250(%rip) contains the constant 1.2f
ucomiss  12(%rsp), %xmm0  ;   12(%rsp) contains the variable b

Notice that the floating-point constant 1.2f did not get embedded into the comparison instruction ucomiss. The constant had to be fetched (with movss) into the register xmm0 before the comparsion could take place. To summarize,

  • a floating-point comparison takes twice as many instructions as an integer comparsion, increasing the executable code size;

  • a floating-point comparison involves an extra fetch instruction (movss), potentially causing a cache miss.

Caveats. As we’ll see in the next section, using integer thresholds will add overhead costs at prediction time. You should ensure that the benefits of integer comparisons outweights the overhead costs.

Mechanism for mapping features

When quantize option is enabled, treelite will collect all thresholds occuring in the tree ensemble model. For each feature, one list will be generated that lists the thresholds in ascending order:

/* example of how per-feature threshold list may look like */

Feature 0:  [1.5, 6.5, 12.5]
Feature 3:  [0.15, 0.35, 1.5]
Feature 6:  [7, 9, 10, 135]

Using these lists, we may convert any data point into integer indices via simple look-ups. For feature 0 in the example above, values will be mapped to integer indices as follows:

Let x be the value of feature 0.

Assign -1 if          x  <  1.5
Assign  0 if          x ==  1.5
Assign  1 if   1.5  < x  <  6.5
Assign  2 if          x ==  6.5
Assign  3 if   6.5  < x  < 12.5
Assign  4 if          x == 12.5
Assign  5 if          x  > 12.5

Let’s look at a specific example of how a floating-point vector gets translated into a vector of integer indices:

feature id   0     1        2      3      4        5      6
            [7, missing, missing, 0.2, missing, missing, 20 ]
         => [3, missing, missing,   1, missing, missing,  5 ]

Since the prediction subroutine still needs to accept floating-point features, the features will be internally converted before actual prediction. If the prediction subroutine looked like below without quantize option,

float predict_margin(const Entry* data) {
  /* ... Run through the trees to compute the leaf output score ... */

  return score;
}

it will now have an extra step of mapping the incoming data vector into integers:

float predict_margin(const Entry* data) {
  /* ... Quantize feature values in data into integer indices   ... */

  /* ... Run through the trees to compute the leaf output score ... */
  return score;
}

Deploying models

After all the hard work you did to train your tree ensemble model, you now have to deploy the model. Deployment refers to distributing your model to other machines and devices so as to make predictions on them. To facilitate the coming discussions, let us define a few terms.

  • Host machine : the machine running treelite.

  • Target machine : the machine on which predictions will be made. The host machine may or may not be identical to the target machine. In cases where it’s infeasible to install treelite on the target machine, the host and target machines will be necessarily distinct.

  • Shared library : a blob of executable subroutines that can be imported by other native applications. Shared libraries will often have file extensions .dll, .so, or .dylib. Going back to the particular context of tree deployment, treelite will produce a shared library containing the prediction subroutine (compiled to native machine code).

  • Runtime package : a tiny fraction of the full treelite package, consisting of a few helper functions that lets you easily load shared libraries and make predictions. The runtime is good to have, but on systems lacking Python we can do without it.

In this document, we will document three options for deployment. We will present the programming interface each deployment option presents, as well as its dependencies and requirements.

Option 1: Install treelite on the target machine

If feasible, this option is probably the most convenient. On the target machine, install treelite by running pip:

pip3 install treelite --user

Once treelite is installed, it suffices to follow instructions in First tutorial.

Dependencies and Requirements

The target machine shall meet the following conditions:

  • Treelite is installed (either by pip install or by manual compilation, see below)

  • One of the following C compiler is available: gcc, clang, Microsoft Visual C++.

  • The C compiler supports the following features of the C99 standard: inline functions; declaration of loop variables inside for loop; the expf function in <math.h>; the <stdint.h> header. Recent versions of gcc and clang should work, as well as Microsoft Visual Studio 2013 or newer.

  • Python is installed, with version 2.7 or >= 3.4.

  • The following Python packages are available: numpy, scipy.sparse.

In addition, if you are using operating systems other than Windows, Mac OS X, and Linux, you would need to compile treelite from the source. To do this, you’ll need git, CMake (>= 3.1), and a C++ compiler that complies with the C++11 standard.

Option 2: Deploy prediction code with the runtime package

[This option is currently not operational. We apologize for inconvenience.]

Option 3: Deploy prediciton code only

With this option, neither Python nor a C++ compiler is required. You should be able to adopt this option using any basic installation of UNIX-like operating systems.

Dependencies and Requirements

The target machine shall meet the following conditions:

  • A C compiler is available.

  • The C compiler supports the following features of the C99 standard: inline functions; declaration of loop variables inside for loop; the expf function in <math.h>; the <stdint.h> header.

  • GNU Make or Microsoft NMake is installed.

  • An archive utility exists that can open a .zip archive.

Deployment instructions

1. On the host machine, install treelite and import your tree ensemble model. You should end up with the model object of type Model.

### Run this block on the **host** machine

import treelite
model = treelite.Model.load('your_model.model', 'xgboost')
# You may also use `from_xgboost` method or the builder class

2. Export your model as a source package by calling the method export_srcpkg() of the Model object. The source package will contain C code representation of the prediction subroutine.

### Continued from the previous code block

# Operating system of the target machine
platform = 'unix'
# C compiler to use to compile prediction code on the target machine
toolchain = 'gcc'
# Save the source package as a zip archive named mymodel.zip
# Later, we'll use this package to produce the library mymodel.so.
model.export_srcpkg(platform=platform, toolchain=toolchain,
                    pkgpath='./mymodel.zip', libname='mymodel.so',
                    verbose=True)

Note

On the value of toolchain

Treelite supports only three toolchain configurations (‘msvc’, ‘gcc’, ‘clang’) for which it generates Makefiles. If you are using a compiler other than these three, you will have to write your own Makefile. For now, just set toolchain='gcc' and move on.

After calling export_srcpkg(), you should be able to find the zip archive named mymodel.zip inside the current working directory.

john.doe@host-machine:/home/john.doe/$ ls .
mymodel.zip   your_model.model

The content of mymodel.zip consists of the header and source files, as well as the Makefile:

john.doe@host-machine:/home/john.doe/$ unzip -l mymodel.zip
Archive:  mymodel.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  11-01-2017 23:11   mymodel/
      167  11-01-2017 23:11   mymodel/Makefile
  4831036  11-01-2017 23:11   mymodel/mymodel.c
      311  11-01-2017 23:11   mymodel/mymodel.h
      109  11-01-2017 23:11   mymodel/recipe.json
---------                     -------
  4831623                     5 files

3. Now you are ready to deploy the model to the target machine. Copy to the target machine the archive mymodel.zip (source package).

john.doe@host-machine:/home/john.doe/$ sftp john.doe@target-machine
Connected to target-machine.
sftp> put mymodel.zip
Uploading mymodel.zip to /home/john.doe/mymodel.zip
mymodel.zip                             100%  410KB 618.2KB/s   00:00
sftp> quit

4. It is time to move to the target machine. On the target machine, extract the archive mymodel.zip:

john.doe@host-machine:/home/john.doe/$ ssh john.doe@target-machine
Last login: Tue Oct 31 00:43:36 2017 from host-machine

john.doe@target-machine:/home/john.doe/$ unzip mymodel.zip
Archive:  mymodel.zip
   creating: mymodel/
  inflating: mymodel/Makefile
  inflating: mymodel/mymodel.c
  inflating: mymodel/mymodel.h
  inflating: mymodel/recipe.json

5. Build the source package (using GNU Make or NMake).

john.doe@target-machine:/home/john.doe/$ cd mymodel
john.doe@target-machine:/home/john.doe/mymodel/$ make
gcc -c -O3 -o mymodel.o mymodel.c -fPIC -std=c99 -flto -fopenmp
gcc -shared -O3 -o mymodel.so mymodel.o -std=c99 -flto -fopenmp
john.doe@target-machine:/home/john.doe/mymodel/$ ls
Makefile       mymodel.c      mymodel.so
mymodel.h      mymodel.o      recipe.json

Note

Parallel compilation with GNU Make

If you used parallel_comp option to split the model into multiple source files, you can take advantage of parallel compilation. Simply replace make with make -jN, where N is replaced with the number of workers to launch. Setting N too high may result into memory shortage.

Note

Using other compilers

If you are using a compiler other than gcc, clang, or Microsoft Visual C++, you will need to compose your own Makefile. Open the Makefile and make necessary changes.

Prediction instructions

The prediction library provides the function predict with the following signature:

float predict(union Entry* data, int pred_margin);

Here, the argument data must be an array of length M, where M is the number of features used in the tree ensemble. The data array stores all the feature values of a single row. To indicate presence or absence of a feature value, we use the union type Entry, which defined as

union Entry {
  int missing;
  float fvalue;
};

For missing values, we set the missing field to -1. For non-missing ones, we set the fvalue field to the feature value. The total number of features is given by the function

size_t get_num_feature(void);

Let’s look at an example. We’d start by initializing the array inst, a dense aray to hold feature values of a single data row:

/* number of features */
const size_t num_feature = get_num_feature();
/* inst: dense vector storing feature values */
union Entry* inst = malloc(sizeof(union Entry) * num_feature);
/* clear inst with all missing values */
for (i = 0; i < num_feature; ++i) {
  inst[i].missing = -1;
}

Before calling the function predict, the array inst needs to be initialized with missing and present feature values. The following peudocode illustrates the idea:

For each data row rid:
  inst[i].missing == -1 for every i, assuming all features lack values

  For each feature i for which the data row in fact has a feature value:
    Set inst[i].fvalue = [feature value], to indicate presence

  Call predict(inst, 0) and get prediction for the data row rid

  For each feature i for which the row has a feature value:
    Set inst[i].missing = -1, to prepare for next row (rid + 1)

The task is not too difficult as long as the input data is given as a particular form of sparse matrix: the Compressed Sparse Row format. The sparse matrix consists of three arrays:

  • val stores nonzero entries in row-major order.

  • col_ind stores column indices of the entries in val. The expression col_ind[i] indicates the column index of the i th entry val[i].

  • row_ptr stores the locations in val that start and end data rows. The i th data row is given by the array slice val[row_ptr[i]:row_ptr[i+1]].

/* nrow : number of data rows */
for (rid = 0; rid < nrow; ++rid) {
  ibegin = row_ptr[rid];
  iend = row_ptr[rid + 1];
  /* Fill nonzeros */
  for (i = ibegin; i < iend; ++i) {
    inst[col_ind[i]].fvalue = val[i];
  }
  out_pred[rid] = predict(inst, 0);
  /* Drop nonzeros */
  for (i = ibegin; i < iend; ++i) {
    inst[col_ind[i]].missing = -1;
  }
}

It only remains to create three arrays val, col_ind, and row_ptr. You may want to use a third-pary library here to read from a SVMLight format. For now, we’ll punt the issue of loading the input data and write it out as constants in the program:

#include <stdio.h>
#include <stdlib.h>
#include "mymodel.h"

int main(void) {
  /* 5x13 "sparse" matrix, in CSR format
     [[ 0.  ,  0.  ,  0.68,  0.99,  0.  ,  0.11,  0.  ,  0.82,  0.  ,
        0.  ,  0.  ,  0.  ,  0.  ],
      [ 0.  ,  0.  ,  0.99,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,
        0.61,  0.  ,  0.  ,  0.  ],
      [ 0.02,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,
        0.  ,  0.  ,  0.  ,  0.  ],
      [ 0.  ,  0.  ,  0.36,  0.  ,  0.82,  0.  ,  0.  ,  0.57,  0.  ,
        0.  ,  0.  ,  0.  ,  0.75],
      [ 0.47,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,
        0.  ,  0.  ,  0.45,  0.  ]]
  */
  const float val[] = {0.68, 0.99, 0.11, 0.82, 0.99, 0.61, 0.02, 0.36, 0.82,
                       0.57, 0.75, 0.47, 0.45};
  const size_t col_ind[] = {2, 3, 5, 7, 2, 9, 0, 2, 4, 7, 12, 0, 11};
  const size_t row_ptr[] = {0, 4, 6, 7, 11, 13};
  const size_t nrow = 5;
  const size_t ncol = 13;

  /* number of features */
  const size_t num_feature = get_num_feature();
  /* inst: dense vector storing feature values */
  union Entry* inst = malloc(sizeof(union Entry) * num_feature);
  float* out_pred = malloc(sizeof(float) * nrow);
  size_t rid, ibegin, iend, i;

  /* clear inst with all missing */
  for (i = 0; i < num_feature; ++i) {
    inst[i].missing = -1;
  }

  for (rid = 0; rid < nrow; ++rid) {
    ibegin = row_ptr[rid];
    iend = row_ptr[rid + 1];
    /* Fill nonzeros */
    for (i = ibegin; i < iend; ++i) {
      inst[col_ind[i]].fvalue = val[i];
    }
    out_pred[rid] = predict(inst, 0);
    /* Drop nonzeros */
    for (i = ibegin; i < iend; ++i) {
      inst[col_ind[i]].missing = -1;
    }
    printf("pred[%zu] = %f\n", rid, out_pred[rid]);
  }
  free(inst);
  free(out_pred);
  return 0;
}

Save the program as a .c file and put it in the same directory mymodel/. To link the program against the prediction library mymodel.so, simply run

gcc -o myprog myprog.c mymodel.so -I. -std=c99

As long as the program myprog is in the same directory of the prediction library mymodel.so, we’ll be good to go.

A sample output:

pred[0] = 44.880001
pred[1] = 44.880001
pred[2] = 44.880001
pred[3] = 42.670002
pred[4] = 44.880001

Deploying models with Treelite4J

Treelite4J is the Java runtime for Treelite. This tutorial will show how to use Treelite4J to deploy decision tree models to Java applications.

Load compiled model

Locate the compiled model (dll/so/dylib) in the local filesystem. We load the compiled model by creating a Predictor object:

import ml.dmlc.treelite4j.Predictor;

Predictor predictor = new Predictor("path/to/compiled_model.so", -1, true, true);

The second argument is set to -1, to utilize all CPU cores available. See here for the meaning of third and fourth arguments.

Query the model

Once the compiled model is loaded, we can query it:

// Get the input dimension, i.e. the number of feature values in the input vector
int num_feature = predictor.GetNumFeature();

// Get the size of output per input
// This number is 1 for tasks other than multi-class classification.
// For multi-class classification task, the number is equal to the number of classes.
int num_output_group = predictor.GetNumOutputGroup();
Predict with a single input (instance)

For predicting with a single input, we create an array of Entry objects, set their values, and invoke the prediction function.

import ml.dmlc.treelite4j.Entry;

// Create an array of feature values for the input
int num_feature = predictor.GetNumFeature();
Entry[] inst = new Entry[num_feature];

// Initialize all feature values as missing
for (int i = 0; i < num_feature; ++i) {
  inst[i] = new Entry();
  inst[i].setMissing();
}

// Set feature values that are not missing
// In this example, we set feature 1, 3, and 7
inst[1].setFValue(-0.5);
inst[3].setFValue(3.2);
inst[7].setFValue(-1.7);

// Now run prediction
// (Put false in the second argument to get probability outputs)
float[] result = predictor.predict(inst, false);
// The result is either class probabilities (for multi-class classification)
// or a single number (for all other tasks, such as regression)
Predict with a batch of inputs

For predicting with a batch of inputs, we create a list of DataPoint objects. Each DataPoint object consists of feature values and corresponding feature indices.

Let us look at an example. Consider the following 4-by-6 data matrix

\left[ \begin{array}{cccccc} 10 & 20 & \cdot & \cdot & \cdot & \cdot\\ \cdot & 30 & \cdot & 40 & \cdot & \cdot\\ \cdot & \cdot & 50 & 60 & 70 & \cdot\\ \cdot & \cdot & \cdot & \cdot & \cdot & 80 \end{array} \right]

where the dot (.) indicates the missing value. The matrix consists of 4 data points (instances), each with 6 feature values. Since not all feature values are present, we need to store feature indices as well as feature values:

import ml.dmlc.treelite4j.DataPoint;

// Create a list consisting of 4 data points
List<DataPoint> dmat = new ArrayList<DataPoint>() {
  {
    //                feature indices     feature values
    add(new DataPoint(new int[]{0, 1},    new float[]{10f, 20f}));
    add(new DataPoint(new int[]{1, 3},    new float[]{30f, 40f}));
    add(new DataPoint(new int[]{2, 3, 4}, new float[]{50f, 60f, 70f}));
    add(new DataPoint(new int[]{5},       new float[]{80f}));
  }
};

Once the list is created, we then convert it into a SparseBatch object. We use SparseBatch rather than DenseBatch because significant portion of the data matrix consists of missing values.

import ml.dmlc.treelite4j.BatchBuilder;

// Convert data point list into SparseBatch object
SparseBatch batch = BatchBuilder.CreateSparseBatch(dmat);

Now invoke the batch prediction function using the SparseBatch object:

// verbose=true, pred_margin=false
float[][] result = predictor.predict(batch, true, false);

The returned array is a two-dimensional array where the array result[i] represents the prediction for the i-th data point. For most applications, each result[i] has length 1. Multi-class classification task is specical, in that for that task result[i] contains class probabilities, so the array is as long as the number of target classes.

For your convenience, we also provide a convenience method to load a data text file in the LIBSVM format:

List<DataPoint> dmat = BatchBuilder.LoadDatasetFromLibSVM("path/to/my.data.libsvm");
SparseBatch batch = BatchBuilder.CreateSparseBatch(dmat);

Specifying models using model builder

Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.

Using XGBoost or LightGBM for training? Read this document instead.

What is the model builder?

The ModelBuilder class is a tool used to specify decision tree ensembles programmatically. Each tree ensemble is represented as follows:

  • Each Tree object is a dictionary of nodes indexed by unique integer keys.

  • A node is either a leaf node or a test node. A test node specifies its left and right children by their integer keys in the tree dictionary.

  • Each ModelBuilder object is a list of Tree objects.

A toy example

Consider the following tree ensemble, consisting of two regression trees:

Note

Provision for missing data: default directions

Decision trees in treelite accomodate missing data by indicating the default direction for every test node. In the diagram above, the default direction is indicated by label “Missing.” For instance, the root node of the first tree shown above will send to the left all data points that lack values for feature 0.

For now, let’s assume that we’ve somehow found optimal choices of default directions at training time. For detailed instructions for actually deciding default directions, see Section 3.4 of the XGBoost paper.

Let us construct this ensemble using the model builder. First step is to assign unique integer key to each node. In the following diagram, integer keys are indicated in red. Note that integer keys need to be unique only within the same tree.

Next, we create a model builder object by calling the constructor for ModelBuilder, with an num_feature argument indicating the total number of features used in the ensemble:

import treelite
builder = treelite.ModelBuilder(num_feature=3)

We also create a tree object; it will represent the first tree in the ensemble.

# to represent the first tree
tree = treelite.ModelBuilder.Tree()

The first tree has five nodes, each of which is to be inserted into the tree one at a time. The syntax for node insertion is as follows:

tree[0]   # insert a new node with key 0

Once a node has been inserted, we can refer to it by writing

tree[0]   # refer to existing node #0

The meaning of the expression tree[0] thus depends on whether the node #0 exists in the tree or not.

We may combine node insertion with a function call to specify its content. For instance, node #0 is a test node, so we call set_numerical_test_node():

# Node #0: feature 0 < 5.0 ? (default direction left)
tree[0].set_numerical_test_node(feature_id=0,
                                opname='<',
                                threshold=5.0,
                                default_left=True,
                                left_child_key=1,
                                right_child_key=2)

On the other hand, node #2 is a leaf node, so call set_leaf_node() instead:

# Node #2: leaf with output +0.6
tree[2].set_leaf_node(0.6)

Let’s go ahead and specify the other three nodes:

# Node #1: feature 2 < -3.0 ? (default direction right)
tree[1].set_numerical_test_node(feature_id=2,
                                opname='<',
                                threshold=-3.0,
                                default_left=False,
                                left_child_key=3,
                                right_child_key=4)
# Node #3: leaf with output -0.4
tree[3].set_leaf_node(-0.4)
# Node #4: leaf with output +1.2
tree[4].set_leaf_node(1.2)

We must indicate which node is the root:

# Set node #0 as root
tree[0].set_root()

We are now done with the first tree. We insert it with the model builder by calling append(). (Recall that the model builder is really a list of tree objects, hence the method name append.)

# Insert the first tree into the ensemble
builder.append(tree)

The second tree is constructed analogously:

tree2 = treelite.ModelBuilder.Tree()
# Node #0: feature 1 < 2.5 ? (default direction right)
tree2[0].set_numerical_test_node(feature_id=1,
                                 opname='<',
                                 threshold=2.5,
                                 default_left=False,
                                 left_child_key=1,
                                 right_child_key=2)
# Set node #0 as root
tree2[0].set_root()
# Node #1: leaf with output +1.6
tree2[1].set_leaf_node(1.6)
# Node #2: feature 2 < -1.2 ? (default direction left)
tree2[2].set_numerical_test_node(feature_id=2,
                                 opname='<',
                                 threshold=-1.2,
                                 default_left=True,
                                 left_child_key=3,
                                 right_child_key=4)
# Node #3: leaf with output +0.1
tree2[3].set_leaf_node(0.1)
# Node #4: leaf with output -0.3
tree2[4].set_leaf_node(-0.3)

# Insert the second tree into the ensemble
builder.append(tree2)

We are now done building the member trees. The last step is to call commit() to finalize the ensemble into a Model object:

# Finalize and obtain Model object
model = builder.commit()

Note

Difference between ModelBuilder and Model objects

Why does treelite require one last step of “committing”? All Model objects are immutable; once constructed, they cannot be modified at all. So you won’t be able to add a tree or a node to an existing Model object, for instance. On the other hand, ModelBuilder objects are mutable, so that you can iteratively build trees.

To ensure we got all details right, we can examine the resulting C program.

model.compile(dirpath='./test')
with open('./test/test.c', 'r') as f:
  for line in f.readlines():
    print(line, end='')

which produces the output

/* Other functions omitted for space consideration */
float predict_margin(union Entry* data) {
  float sum = 0.0f;
  if (!(data[0].missing != -1) || data[0].fvalue < 5) {
    if ( (data[2].missing != -1) && data[2].fvalue < -3) {
      sum += (float)-0.4;
    } else {
      sum += (float)1.2;
    }
  } else {
    sum += (float)0.6;
  }
  if ( (data[1].missing != -1) && data[1].fvalue < 2.5) {
    sum += (float)1.6;
  } else {
    if (!(data[2].missing != -1) || data[2].fvalue < -1.2) {
      sum += (float)0.1;
    } else {
      sum += (float)-0.3;
    }
  }
  return sum + (0);
}

The toy example has been helpful as an illustration, but it is impractical to manually specify nodes for real-world ensemble models. The following section will show us how to automate the tree building process. We will look at scikit-learn in particular.

Using the model builder to interface with scikit-learn

Scikit-learn (scikit-learn/scikit-learn) is a Python machine learning package known for its versatility and ease of use. It supports a wide variety of models and algorithms.

Treelite will be able to work with any decision tree ensemble models produced by scikit-learn. In particular, it will be able to work with

Note

Why scikit-learn? How about other packages?

We had to pick a specific example for programmatic tree construction, so we chose scikit-learn. If you’re using another package, don’t lose heart. As you read through the rest of section, notice how specific pieces of information about the tree ensemble model are being extracted. As long as your choice of package exposes equivalent information, you’ll be able to adapt the example to your needs.

Note

In a hurry? Try the gallery module

The rest of this document explains in detail how to import scikit-learn models using the builder class. If you prefer to skip all the gory details, simply import the module treelite.gallery.sklearn.

import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)

Note

Adaboost ensembles not yet supported

Treelite currently does not support weighting of member trees, so you won’t be able to use Adaboost ensembles.

Regression with RandomForestRegressor

Let’s start with the Boston house prices dataset, a regression problem. (Classification problems are somewhat trickier, so we’ll save them for later.)

We’ll be using RandomForestRegressor, a random forest for regression. A random forest is an ensemble of decision trees that are independently trained on random samples from the training data. See this page for more details. For now, just remember to specify random_forest=True in the ModelBuilder constructor.

import sklearn.datasets
import sklearn.ensemble
# Load the Boston housing dataset
X, y = sklearn.datasets.load_boston(return_X_y=True)
# Train a random forest regressor with 10 trees
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)

We shall programmatically construct Tree objects from internal attributes of the scikit-learn model. We only need to define a few helper functions.

For the rest of sections, we’ll be diving into lots of details that are specific to scikit-learn. Many details have been adopted from this reference page.

The function process_model() takes in a scikit-learn ensemble object and returns the completed Model object:

def process_model(sklearn_model):
  # Initialize treelite model builder
  # Set random_forest=True for random forests
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  random_forest=True)

  # Iterate over individual trees
  for i in range(sklearn_model.n_estimators):
    # Process the i-th tree and add to the builder
    # process_tree() to be defined later
    builder.append( process_tree(sklearn_model.estimators_[i].tree_,
                                 sklearn_model) )

  return builder.commit()

The usage of this function is as follows:

model = process_model(clf)

We won’t have space here to discuss all internals of scikit-learn objects, but a few details should be noted:

  • The attribute n_features_ stores the number of features used anywhere in the tree ensemble.

  • The attribute n_estimators stores the number of member trees.

  • The attribute estimators_ is an array of handles that store the individual member trees. To access the object for the i-th tree, write estimators_[i].tree_. This object will be passed to the function process_tree().

The function process_tree() takes in a single scikit-learn tree object and returns an object of type Tree:

def process_tree(sklearn_tree, sklearn_model):
  treelite_tree = treelite.ModelBuilder.Tree()
  # Node #0 is always root for scikit-learn decision trees
  treelite_tree[0].set_root()

  # Iterate over each node: node ID ranges from 0 to [node_count]-1
  for node_id in range(sklearn_tree.node_count):
    process_node(treelite_tree, sklearn_tree, node_id, sklearn_model)

  return treelite_tree

Explanations:

  • The attribute node_count stores the number of nodes in the decision tree.

  • Each node in the tree has a unique ID ranging from 0 to [node_count]-1.

The function process_node() determines whether each node is a leaf node or a test node. It does so by looking at the attribute children_left: If the left child of the node is set to -1, that node is thought to be a leaf node.

def process_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  if sklearn_tree.children_left[node_id] == -1:  # leaf node
    process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model)
  else:                                          # test node
    process_test_node(treelite_tree, sklearn_tree, node_id, sklearn_model)

The function process_test_node() extracts the content of a test node and passes it to the Tree object that is being constructed.

def process_test_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  # Initialize the test node with given node ID
  treelite_tree[node_id].set_numerical_test_node(
                        feature_id=sklearn_tree.feature[node_id],
                        opname='<=',
                        threshold=sklearn_tree.threshold[node_id],
                        default_left=True,
                        left_child_key=sklearn_tree.children_left[node_id],
                        right_child_key=sklearn_tree.children_right[node_id])

Explanations:

  • The attribute feature is the array containing feature indices used in test nodes.

  • The attribute threshold is the array containing threshold values used in test nodes.

  • All tests are in the form of [feature value] <= [threshold].

  • The attributes children_left and children_right together store children’s IDs for test nodes.

Note

Scikit-learn and missing data

Scikit-learn handles missing data differently than XGBoost and treelite. Before training an ensemble model, you’ll have to impute missing values. For this reason, test nodes in scikit-learn tree models will contain no “default direction.” We will assign default_left=True arbitrarily for test nodes to keep treelite happy.

The function process_leaf_node() defines a leaf node:

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  # The `value` attribute stores the output for every leaf node.
  leaf_value = sklearn_tree.value[node_id].squeeze()
  # Initialize the leaf node with given node ID
  treelite_tree[node_id].set_leaf_node(leaf_value)

Let’s test it out:

model = process_model(clf)
model.export_lib(libpath='./libtest.dylib', toolchain='gcc', verbose=True)

import treelite.runtime
predictor = treelite.runtime.Predictor(libpath='./libtest.dylib')
predictor.predict(treelite.runtime.Batch.from_npy2d(X))
Regression with GradientBoostingRegressor

Gradient boosting is an algorithm where decision trees are trained one at a time, ensuring that latter trees complement former trees. See this page for more details. Treelite makes distinction between random forests and gradient boosted trees by the value of random_forest flag in the ModelBuilder constructor.

Note

Set init='zero' to ensure compatibility

To make sure that the gradient boosted model is compatible with treelite, make sure to set init='zero' in the GradientBoostingRegressor constructor. This ensures that the compiled prediction subroutine will produce the correct prediction output. Gradient boosting models trained without specifying init='zero' in the constructor are NOT supported by treelite!

# Gradient boosting regressor
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingRegressor(n_estimators=10,
                                                 init='zero')
clf.fit(X, y)

We will recycle most of the helper code we wrote earlier. Only two functions will need to be modified:

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  # Check for init='zero'
  if sklearn_model.init != 'zero':
    raise Exception("Gradient boosted trees must be trained with "
                    "the option init='zero'")
  # Initialize treelite model builder
  # Set random_forest=False for gradient boosted trees
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  random_forest=False)
  for i in range(sklearn_model.n_estimators):
    # Process i-th tree and add to the builder
    builder.append( process_tree(sklearn_model.estimators_[i][0].tree_,
                                 sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  leaf_value = sklearn_tree.value[node_id].squeeze()
  # Need to shrink each leaf output by the learning rate
  leaf_value *= sklearn_model.learning_rate
  # Initialize the leaf node with given node ID
  treelite_tree[node_id].set_leaf_node(leaf_value)

Some details specific to GradientBoostingRegressor:

  • To indicate the use of gradient boosting (as opposed to random forests), we set random_forest=False in the ModelBuilder constructor.

  • Each tree object is now accessed with the expression estimators_[i][0].tree_, as estimators_[i] returns an array consisting of a single tree (i-th tree).

  • Each leaf output in gradient boosted trees are “unscaled”: it needs to be scaled by the learning rate.

Let’s test it:

# Convert to treelite model
model = process_model(clf)
# Generate shared library
model.export_lib(libpath='./libtest2.dylib', toolchain='gcc', verbose=True)
# Make prediction with predictor
predictor = treelite.runtime.Predictor(libpath='./libtest2.dylib')
predictor.predict(treelite.runtime.Batch.from_npy2d(X))
Binary Classification with RandomForestClassifier

For binary classification, let’s use the digits dataset. We will take 0’s and 1’s from the dataset and treat 0’s as the negative class and 1’s as the positive.

# load a binary classification problem
# Set n_class=2 to produce two classes
digits = sklearn.datasets.load_digits(n_class=2)
X, y = digits['data'], digits['target']
# Should print [0 1]
print(np.unique(y))

# Train a random forest classifier
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)

Random forest classifiers in scikit-learn store frequency counts for the positive and negative class. For instance, a leaf node may output a set of counts

[ 100, 200 ]

which indicates the following:

  • 300 data points in the training set “belong” to this leaf node, in the sense that they all satisfy the precise sequence of conditions leading to that particular leaf node. The picture below shows that each leaf node represents a unique sequence of conditions:

  • 100 of them are labeled negative; and

  • the remaining 200 are labeled positive.

Again, most of the helper functions may be re-used; only two functions need to be rewritten. Explanation will follow after the code:

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  random_forest=True)
  for i in range(sklearn_model.n_estimators):
    # Process i-th tree and add to the builder
    builder.append( process_tree(sklearn_model.estimators_[i].tree_,
                                 sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  # Get counts for each label (+/-) at this leaf node
  leaf_count = sklearn_tree.value[node_id].squeeze()
  # Compute the fraction of positive data points at this leaf node
  fraction_positive = float(leaf_count[1]) / leaf_count.sum()
  # The fraction above is now the leaf output
  treelite_tree[node_id].set_leaf_node(fraction_positive)

As noted earlier, we access the frequency counts at each leaf node, reading the value attribute of each tree. Then we compute the fraction of positive data points with respect to all training data points belonging to the leaf. This fraction then becomes the leaf output. This way, leaf nodes now produce single numbers rather than frequency count arrays.

Why did we have to compute a fraction? For binary classification, treelite expects each tree to produce a single number output. At prediction time, the outputs from the member trees will get averaged to produce the final prediction, which is also a single number. By setting the positive fraction as the leaf output, we ensure that the final prediction is a proper probability value. For instance, if an ensemble consisting of 5 trees produces the following set of outputs

Tree 0    0.1
Tree 1    0.7
Tree 2    0.4
Tree 3    0.3
Tree 4    0.7

then the final prediction will be 0.44, which we interpret as 44% probability for the positive class.

Multi-class Classification with RandomForestClassifier

Let’s use the digits dataset again, this time with 4 classes (i.e. 0’s, 1’s, 2’s, and 3’s).

# Load a multi-class classification problem
# Set n_class=4 to produce four classes
digits = sklearn.datasets.load_digits(n_class=4)
X, y = digits['data'], digits['target']
# Should print [0 1 2 3]
print(np.unique(y))

# Train a random forest classifier
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)

Random forest classifiers in scikit-learn store frequency counts (see the explanation in the previous section). For instance, a leaf node may output a set of counts

[ 100, 400, 300, 200 ]

which shows that the total of 1000 training data points belong to this leaf node and that 100, 400, 300, and 200 of them are labeled class 0, 1, 2, and 3, respectively.

We will have to re-write the process_leaf_node() function to accomodate multiple classes.

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  # Must specify num_output_group and pred_transform
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  num_output_group=sklearn_model.n_classes_,
                                  random_forest=True,
                                  pred_transform='identity_multiclass')
  for i in range(sklearn_model.n_estimators):
    # Process i-th tree and add to the builder
    builder.append( process_tree(sklearn_model.estimators_[i].tree_,
                                 sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  # Get counts for each label class at this leaf node
  leaf_count = sklearn_tree.value[node_id].squeeze()
  # Compute the probability distribution over label classes
  prob_distribution = leaf_count / leaf_count.sum()
  # The leaf output is the probability distribution
  treelite_tree[node_id].set_leaf_node(prob_distribution)

The process_leaf_node() function is quite similar to what we had for the binary classification case. Only difference is that, instead of computing the fraction of the positive class, we compute the probability distribution for all possible classes. Each leaf node thus will store the probability distribution of possible class outcomes.

The process_model() function is also similar to what we had before. The crucial difference is the existence of parameters num_output_group and pred_transform. The num_output_group parameter is used only for multi-class classification: it should store the number of classes (in this example, 4). The pred_transform parameter should be set to 'identity_multiclass', to indicate that the prediction should be made simply by averaging the probability distribution produced by each leaf node. (Leaf outputs are averaged rather than summed because we set random_forest=True.) For instance, if an ensemble consisting of 3 trees produces the following set of outputs

Tree 0    [ 0.5, 0.5, 0.0, 0.0 ]
Tree 1    [ 0.1, 0.5, 0.3, 0.1 ]
Tree 2    [ 0.2, 0.5, 0.2, 0.1 ]

then the final prediction will be the average [ 0.26666667, 0.5, 0.16666667, 0.06666667 ], which indicates 26.7% probability for the first class, 50.0% for the second, 16.7% for the third, and 6.7% for the fourth.

Binary Classification with GradientBoostingClassifier

We use the digits dataset. We will take 0’s and 1’s from the dataset and treat 0’s as the negative class and 1’s as the positive.

# Load a binary classification problem
# Set n_class=2 to produce two classes
digits = sklearn.datasets.load_digits(n_class=2)
X, y = digits['data'], digits['target']
# Should print [0 1]
print(np.unique(y))

# Train a gradient boosting classifier
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingClassifier(n_estimators=10,
                                                  init='zero')
clf.fit(X, y)

Note

Set init='zero' to ensure compatibility

To make sure that the gradient boosted model is compatible with treelite, make sure to set init='zero' in the GradientBoostingClassifier constructor. This ensures that the compiled prediction subroutine will produce the correct prediction output. Gradient boosting models trained without specifying init='zero' in the constructor are NOT supported by treelite!

Here are the functions process_model() and process_leaf_node() for this scenario:

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  # Check for init='zero'
  if sklearn_model.init != 'zero':
    raise Exception("Gradient boosted trees must be trained with "
                    "the option init='zero'")
  # Initialize treelite model builder
  # Set random_forest=False for gradient boosted trees
  # Set pred_transform='sigmoid' to obtain probability predictions
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  random_forest=False,
                                  pred_transform='sigmoid')
  for i in range(sklearn_model.n_estimators):
    # Process i-th tree and add to the builder
    builder.append( process_tree(sklearn_model.estimators_[i][0].tree_,
                                 sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  leaf_value = sklearn_tree.value[node_id].squeeze()
  # Need to shrink each leaf output by the learning rate
  leaf_value *= sklearn_model.learning_rate
  # Initialize the leaf node with given node ID
  treelite_tree[node_id].set_leaf_node(leaf_value)

Some details specific to GradientBoostingClassifier:

  • To indicate the use of gradient boosting (as opposed to random forests), we set random_forest=False in the ModelBuilder constructor.

  • Each tree object is now accessed with the expression estimators_[i][0].tree_, as estimators_[i] returns an array consisting of a single tree (i-th tree).

  • Each leaf output in gradient boosted trees are “unscaled”: it needs to be scaled by the learning rate.

In addition, we specify the parameter pred_transform='sigmoid' so that the final prediction yields the probability for the positive class. For example, suppose that an ensemble consisting of 4 trees produces the following set of outputs:

Tree 0    +0.5
Tree 1    -2.3
Tree 2    +1.5
Tree 3    -1.5

Unlike the random forest example earlier, we do not assume that each leaf output is between 0 and 1; it can be any real number, negative or positive. These numbers are referred to as margin scores, to distinguish them from probabilities.

To obtain the probability for the positive class, we first sum the margin scores (outputs) from the member trees.

Tree 0    +0.5
Tree 1    -2.3
Tree 2    +1.5
Tree 3    -1.5
--------------
Total     -1.8

Then we apply the sigmoid function:

\sigma(x) = \frac{1}{1 + e^{-x}}

The resulting value is the final prediction. You may interpret this value as a probability. For the particular example, the sigmoid value of -1.8 is 0.14185106, which we interpret as 14.2% probability for the positive class.

Multi-class Classification with GradientBoostingClassifier

Let’s use the digits dataset again, this time with 4 classes (i.e. 0’s, 1’s, 2’s, and 3’s).

# Load a multi-class classification problem
# Set n_class=4 to produce four classes
digits = sklearn.datasets.load_digits(n_class=4)
X, y = digits['data'], digits['target']
# Should print [0 1 2 3]
print(np.unique(y))

# Train a gradient boosting classifier
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingClassifier(n_estimators=10,
                                                  init='zero')
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)

Note

Set init='zero' to ensure compatibility

To make sure that the gradient boosted model is compatible with treelite, make sure to set init='zero' in the GradientBoostingClassifier constructor. This ensures that the compiled prediction subroutine will produce the correct prediction output. Gradient boosting models trained without specifying init='zero' in the constructor are NOT supported by treelite!

Here are the functions process_model() and process_leaf_node() for this scenario:

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  # Check for init='zero'
  if sklearn_model.init != 'zero':
    raise Exception("Gradient boosted trees must be trained with "
                    "the option init='zero'")
  # Initialize treelite model builder
  # Set random_forest=False for gradient boosted trees
  # Set num_output_group for multiclass classification
  # Set pred_transform='softmax' to obtain probability predictions
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  num_output_group=sklearn_model.n_classes_,
                                  random_forest=False,
                                  pred_transform='softmax')
  # Process [number of iterations] * [number of classes] trees
  for i in range(sklearn_model.n_estimators):
    for k in range(sklearn_model.n_classes_):
      builder.append( process_tree(sklearn_model.estimators_[i][k].tree_,
                                   sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  leaf_value = sklearn_tree.value[node_id].squeeze()
  # Need to shrink each leaf output by the learning rate
  leaf_value *= sklearn_model.learning_rate
  # Initialize the leaf node with given node ID
  treelite_tree[node_id].set_leaf_node(leaf_value)

The process_leaf_node() function is identical to one in the previous section: as before, each leaf node produces a single real-number output.

On the other hand, the process_model() function needs some explanation. First of all, the attribute estimators_ of the scikit-learn model object now stores output groups, which are simply groups of decision trees. The expression estimators_[i] thus refers to the i th output group. Each output group contains as many trees as there are label classes. For the digits example with 4 label classes, we’d have 4 trees for each output group: estimators_[i][0], estimators_[i][1], estimators_[i][2], and estimators_[i][3]. Since there are as many output groups as the number of iterations used for training, the total number of member trees is [number of iterations] * [number of classes]. We have to call append() once for each member tree; hence the use of nested loop.

We also set pred_transform='softmax', which indicates the way margin outputs should be transformed to produce probability predictions. Let us look at a concrete example: suppose we train an ensemble model with 3 rounds of gradient boosting. It would produce a total of 12 decision trees (3 rounds * 4 classes). Suppose also that, given a single test data point, the model produces the following set of margins:

Output group 0:
  Tree  0 produces  +0.5
  Tree  1 produces  +1.5
  Tree  2 produces  -2.3
  Tree  3 produces  -1.5
Output group 1:
  Tree  4 produces  +0.1
  Tree  5 produces  +0.7
  Tree  6 produces  +1.5
  Tree  7 produces  -0.9
Output group 2:
  Tree  8 produces  -0.1
  Tree  9 produces  +0.3
  Tree 10 produces  -0.7
  Tree 11 produces  +0.2

How do we compute probabilities for each of the 4 classes? First, we compute the sum of the margin scores for each output group:

Output group 0:
  Tree  0 produces  +0.5
  Tree  1 produces  +1.5
  Tree  2 produces  -2.3
  Tree  3 produces  -1.5
  ----------------------
  SUBTOTAL          -1.8
Output group 1:
  Tree  4 produces  +0.1
  Tree  5 produces  +0.7
  Tree  6 produces  +1.5
  Tree  7 produces  -0.9
  ----------------------
  SUBTOTAL          +1.4
Output group 2:
  Tree  8 produces  -0.1
  Tree  9 produces  +0.3
  Tree 10 produces  -0.7
  Tree 11 produces  +0.2
  ----------------------
  SUBTOTAL          -0.3

The vector [-1.8, +1.4, -0.3] consisting of the subtotals quantifies the relative likelihood of the label classes. Since the second element (1.4) is the largest, the second class must be the most likely outcome for the particular data point. This vector is not yet a probability distribution, since its elements do not sum to 1.

The softmax function transforms any real-valued vector into a probability distribution as follows:

  1. Apply the exponential function (exp) to every element in the vector. This step ensures that every element is positive.

  2. Divide every element by the sum over the vector. This step is also known as normalizing the vector. After thie step, the elements of the vector will add up to 1.

Let’s walk through the steps with the vector [-1.8, +1.4, -0.3]. Applying the exponential function is simple with Python:

x = np.exp([-1.8, +1.4, -0.3])
print(x)

which yields

[ 0.16529889  4.05519997  0.74081822]

Note that every element is now positive. Then we normalize the vector by writing

x = x / x.sum()
print(x)

which gives a proper probability distribution:

[ 0.03331754  0.8173636   0.14931886]

We can now interpret the result as giving 3.3% probability for the first class, 81.7% probability for the second, and 14.9% probability for the third.

Specifying models using Protocol Buffers

Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.

Using XGBoost or LightGBM for training? Read this document instead.

What is Protocol Buffers?

Protocol Buffers (google/protobuf) is a widely used mechanism to serialize structured data. You may specify your ensemble model according to the specification src/tree.proto. Depending on the package you used to train the model, it may take some effort to express the model in terms of the given spec. See this helpful guide on reading and writing serialized messages.

To import models that had been serialized with Protocol Buffers, use the load() method with argument format='protobuf':

# model had been saved to a file named my_model.bin
# notice the second argument format='protobuf'
model = Model.load('my_model.bin', format='protobuf')

Treelite API

API of Treelite Python package.

Treelite: a model compiler for decision tree ensembles

class treelite.DMatrix(data, data_format=None, missing=None, feature_names=None, feature_types=None, verbose=False, nthread=None)

Data matrix used in treelite.

Parameters

  • data (str / numpy.ndarray / scipy.sparse.csr_matrix / pandas.DataFrame) – Data source. When data is str type, it indicates that data should be read from a file.

  • data_format (str, optional) – Format of input data file. Applicable only when data is read from a file. If missing, the svmlight (.libsvm) format is assumed.

  • missing (float, optional) – Value in the data that represents a missing entry. If set to None, numpy.nan will be used.

  • verbose (bool, optional) – Whether to print extra messages during construction

  • feature_names (list, optional) – Human-readable names for features

  • feature_types (list, optional) – Types for features

  • nthread (int, optional) – Number of threads

class treelite.Model(handle=None)

Decision tree ensemble model

Parameters

handle (ctypes.c_void_p, optional) – Initial value of model handle

compile(dirpath, params=None, compiler='ast_native', verbose=False)

Generate prediction code from a tree ensemble model. The code will be C99 compliant. One header file (.h) will be generated, along with one or more source files (.c). Use create_shared() method to package prediction code as a dynamic shared library (.so/.dll/.dylib).

Parameters

  • dirpath (str) – directory to store header and source files

  • params (dict, optional) – parameters for compiler. See this page for the list of compiler parameters.

  • compiler (str, optional) – name of compiler to use

  • verbose (bool, optional) – Whether to print extra messages during compilation

Example

The following populates the directory ./model with source and header files:

model.compile(dirpath='./my/model', params={}, verbose=True)

If parallel compilation is enabled (parameter parallel_comp), the files are in the form of ./my/model/header.h, ./my/model/main.c, ./my/model/tu0.c, ./my/model/tu1.c and so forth, depending on the value of parallel_comp. Otherwise, there will be exactly two files: ./model/header.h, ./my/model/main.c

export_lib(toolchain, libpath, params=None, compiler='ast_native', verbose=False, nthread=None, options=None)

Convenience function: Generate prediction code and immediately turn it into a dynamic shared library. A temporary directory will be created to hold the source files.

Parameters

  • toolchain (str) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’. You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)

  • libpath (str) – location to save the generated dynamic shared library

  • params (dict, optional) – parameters to be passed to the compiler. See this page for the list of compiler parameters.

  • compiler (str, optional) – name of compiler to use in C code generation

  • verbose (bool, optional) – whether to produce extra messages

  • nthread (int, optional) – number of threads to use in creating the shared library. Defaults to the number of cores in the system.

  • options (list of str, optional) – Additional options to pass to toolchain

Example

The one-line command

model.export_lib(toolchain='msvc', libpath='./mymodel.dll',
                 params={}, verbose=True)

is equivalent to the following sequence of commands:

model.compile(dirpath='/temporary/directory', params={}, verbose=True)
treelite.create_shared(toolchain='msvc', dirpath='/temporary/directory',
              verbose=True)
# move the library out of the temporary directory
shutil.move('/temporary/directory/mymodel.dll', './mymodel.dll')
export_protobuf(filename)

Export a tree ensemble model as a Protocol Buffers format. Protocol Buffers (google/protobuf) is a language- and platform-neutral mechanism for serializing structured data. See src/tree.proto for format spec.

Parameters

filename (str) – path to save Protocol Buffers output

Example

model.export_protobuf('./my.buffer')
export_srcpkg(platform, toolchain, pkgpath, libname, params=None, compiler='ast_native', verbose=False, options=None)

Convenience function: Generate prediction code and create a zipped source package for deployment. The resulting zip file will also contain a Makefile.

Parameters

  • platform (str) – name of the operating system on which the headers and sources shall be compiled. Must be one of the following: ‘windows’ (Microsoft Windows), ‘osx’ (Mac OS X), ‘unix’ (Linux and other UNIX-like systems)

  • toolchain (str) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’. You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)

  • pkgpath (str) – location to save the zipped source package

  • libname (str) – name of model shared library to be built

  • params (dict, optional) – parameters to be passed to the compiler. See this page for the list of compiler parameters.

  • compiler (str, optional) – name of compiler to use in C code generation

  • verbose (bool, optional) – whether to produce extra messages

  • nthread (int, optional) – number of threads to use in creating the shared library. Defaults to the number of cores in the system.

  • options (list of str, optional) – Additional options to pass to toolchain

Example

The one-line command

model.export_srcpkg(platform='unix', toolchain='gcc',
                    pkgpath='./mymodel_pkg.zip', libname='mymodel.so',
                    params={}, verbose=True)

is equivalent to the following sequence of commands:

model.compile(dirpath='/temporary/directory/mymodel',
              params={}, verbose=True)
generate_makefile(dirpath='/temporary/directory/mymodel',
                  platform='unix', toolchain='gcc')
# zip the directory containing C code and Makefile
shutil.make_archive(base_name=pkgpath, format='zip',
                    root_dir='/temporary/directory',
                    base_dir='mymodel/')
classmethod from_xgboost(booster)

Load a tree ensemble model from an XGBoost Booster object

Parameters

booster (object of type xgboost.Booster) – Python handle to XGBoost model

Returns

model – loaded model

Return type

Model object

Example

bst = xgboost.train(params, dtrain, 10, [(dtrain, 'train')])
xgb_model = Model.from_xgboost(bst)
classmethod load(filename, model_format)

Load a tree ensemble model from a file

Parameters

  • filename (str) – path to model file

  • model_format (str) – model file format. Must be one or ‘xgboost’, ‘lightgbm’, ‘protobuf’

Returns

model – loaded model

Return type

Model object

Example

xgb_model = Model.load('xgboost_model.model', 'xgboost')
property num_feature

Number of features used in the model

property num_output_group

Number of output groups of the model

property num_tree

Number of decision trees in the model

set_tree_limit(n)

Set first n trees to be kept, the remaining ones will be dropped

class treelite.ModelBuilder(num_feature, num_output_group=1, random_forest=False, **kwargs)

Builder class for tree ensemble model: provides tools to iteratively build an ensemble of decision trees

Parameters

  • num_feature (int) – number of features used in model being built. We assume that all feature indices are between 0 and (num_feature - 1)

  • num_output_group (int, optional) – number of output groups; >1 indicates multiclass classification

  • random_forest (bool, optional) – whether the model is a random forest; True indicates a random forest and False indicates gradient boosted trees

  • **kwargs – model parameters, to be used to specify the resulting model. Refer to this page for the full list of model parameters.

class Node

Handle to a node in a tree

set_categorical_test_node(feature_id, left_categories, default_left, left_child_key, right_child_key)

Set the node as a test node with categorical split. A list defines all categories that would be classified as the left side. Categories are integers ranging from 0 to n-1, where n is the number of categories in that particular feature.

Parameters

  • feature_id (int) – feature index

  • left_categories (list of int) – list of categories belonging to the left child.

  • default_left (bool) – default direction for missing values (True for left; False for right)

  • left_child_key (int) – unique integer key to identify the left child node

  • right_child_key (int) – unique integer key to identify the right child node

set_leaf_node(leaf_value)

Set the node as a leaf node

Parameters

leaf_value (float / list of float) – Usually a single leaf value (weight) of the leaf node. For multiclass random forest classifier, leaf_value should be a list of leaf weights.

set_numerical_test_node(feature_id, opname, threshold, default_left, left_child_key, right_child_key)

Set the node as a test node with numerical split. The test is in the form [feature value] OP [threshold]. Depending on the result of the test, either left or right child would be taken.

Parameters

  • feature_id (int) – feature index

  • opname (str) – binary operator to use in the test

  • threshold (float) – threshold value

  • default_left (bool) – default direction for missing values (True for left; False for right)

  • left_child_key (int) – unique integer key to identify the left child node

  • right_child_key (int) – unique integer key to identify the right child node

set_root()

Set the node as the root

class Tree

Handle to a decision tree in a tree ensemble Builder

append(tree)

Add a tree at the end of the ensemble

Parameters

tree (Tree object) – tree to be added

Example

builder = ModelBuilder(num_feature=4227)
tree = ...               # build tree somehow
builder.append(tree)     # add tree at the end of the ensemble
commit()

Finalize the ensemble model

Returns

model – finished model

Return type

Model object

Example

builder = ModelBuilder(num_feature=4227)
for i in range(100):
  tree = ...                    # build tree somehow
  builder.append(tree)          # add one tree at a time

model = builder.commit()        # now get a Model object
model.compile(dirpath='test')   # compile model into C code
insert(index, tree)

Insert a tree at specified location in the ensemble

Parameters

  • index (int) – index of the element before which to insert the tree

  • tree (Tree object) – tree to be inserted

Example

builder = ModelBuilder(num_feature=4227)
tree = ...               # build tree somehow
builder.insert(0, tree)  # insert tree at index 0
class treelite.Annotator

Branch annotator class: annotate branches in a given model using frequency patterns in the training data

annotate_branch(model, dmat, nthread=None, verbose=False)

Annotate branches in a given model using frequency patterns in the training data. Each node gets the count of the instances that belong to it. Any prior annotation information stored in the annotator will be replaced with the new annotation returned by this method.

Parameters

  • model (object of type Model) – decision tree ensemble model

  • dmat (object of type DMatrix) – data matrix representing the training data

  • nthread (int, optional) – number of threads to use while annotating. If missing, use all physical cores in the system.

  • verbose (bool, optional) – whether to produce extra messages

save(path)

Save branch annotation infromation as a JSON file.

Parameters

path (str) – location of saved JSON file

treelite.create_shared(toolchain, dirpath, nthread=None, verbose=False, options=None)

Create shared library.

Parameters

  • toolchain (str) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’. You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)

  • dirpath (str) – directory containing the header and source files previously generated by Model.compile(). The directory must contain recipe.json which specifies build dependencies.

  • nthread (int, optional) – number of threads to use in creating the shared library. Defaults to the number of cores in the system.

  • verbose (bool, optional) – whether to produce extra messages

  • options (list of str, optional) – Additional options to pass to toolchain

Returns

libpath – absolute path of created shared library

Return type

str

Example

The following command uses Visual C++ toolchain to generate ./my/model/model.dll:

model.compile(dirpath='./my/model', params={}, verbose=True)
create_shared(toolchain='msvc', dirpath='./my/model', verbose=True)

Later, the shared library can be referred to by its directory name:

predictor = Predictor(libpath='./my/model', verbose=True)
# looks for ./my/model/model.dll

Alternatively, one may specify the library down to its file name:

predictor = Predictor(libpath='./my/model/model.dll', verbose=True)
treelite.save_runtime_package(destdir)

Save a copy of the (zipped) runtime package, containing all glue code necessary to deploy compiled models into the wild

Parameters

destdir (str) – directory to save the zipped package

treelite.generate_makefile(dirpath, platform, toolchain, options=None)

Generate a Makefile for a given directory of headers and sources. The resulting Makefile will be stored in the directory. This function is useful for deploying a model on a different machine.

Parameters

  • dirpath (str) – directory containing the header and source files previously generated by Model.compile(). The directory must contain recipe.json which specifies build dependencies.

  • platform (str) – name of the operating system on which the headers and sources shall be compiled. Must be one of the following: ‘windows’ (Microsoft Windows), ‘osx’ (Mac OS X), ‘unix’ (Linux and other UNIX-like systems)

  • toolchain (str) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’. You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)

  • options (list of str, optional) – Additional options to pass to toolchain

treelite.gallery.sklearn.import_model(sklearn_model)

Load a tree ensemble model from a scikit-learn model object

Parameters

sklearn_model (object of type RandomForestRegressor / RandomForestClassifier / GradientBoostingRegressor / GradientBoostingClassifier) – Python handle to scikit-learn model

Returns

model – loaded model

Return type

Model object

Example

import sklearn.datasets
import sklearn.ensemble
X, y = sklearn.datasets.load_boston(return_X_y=True)
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)

import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)

Treelite runtime API

Runtime API of treelite Python package.

Treelite prediction runtime package

class treelite.runtime.Predictor(libpath, nthread=None, verbose=False)

Predictor class: loader for compiled shared libraries

Parameters

  • libpath (str) – location of dynamic shared library (.dll/.so/.dylib)

  • nthread (int, optional) – number of worker threads to use; if unspecified, use maximum number of hardware threads

  • verbose (bool, optional) – Whether to print extra messages during construction

property global_bias

Query global bias of the model

property num_feature

Query number of features used in the model

property num_output_group

Query number of output groups of the model

property pred_transform

Query pred transform of the model

predict(batch, verbose=False, pred_margin=False)

Perform batch prediction with a 2D sparse data matrix. Worker threads will internally divide up work for batch prediction. Note that this function may be called by only one thread at a time. In order to use multiple threads to process multiple prediction requests simultaneously, use predict_instance() instead.

Parameters

  • batch (object of type Batch) – batch of rows for which predictions will be made

  • verbose (bool, optional) – Whether to print extra messages during prediction

  • pred_margin (bool, optional) – whether to produce raw margins rather than transformed probabilities

predict_instance(inst, missing=None, pred_margin=False)

Perform single-instance prediction. Prediction is run by the calling thread.

Parameters

  • inst (numpy.ndarray / scipy.sparse.csr_matrix / dict) – Data instance for which a prediction will be made. If inst is of type scipy.sparse.csr_matrix, its first dimension must be 1 (shape[0]==1). If inst is of type numpy.ndarray, it must be one-dimensional. If inst is of type dict, it must be a dictionary where the keys indicate feature indices (0-based) and the values corresponding feature values.

  • missing (float, optional) – Value in the data instance that represents a missing value. If set to None, numpy.nan will be used. Only applicable if inst is of type numpy.ndarray.

  • pred_margin (bool, optional) – Whether to produce raw margins rather than transformed probabilities

property sigmoid_alpha

Query sigmoid alpha of the model

class treelite.runtime.Batch

Batch of rows to be used for prediction

classmethod from_csr(csr, rbegin=None, rend=None)

Get a sparse batch from a subset of rows in a CSR (Compressed Sparse Row) matrix. The subset is given by the range [rbegin, rend).

Parameters

  • csr (object of class treelite.DMatrix or scipy.sparse.csr_matrix) – data matrix

  • rbegin (int, optional) – the index of the first row in the subset

  • rend (int, optional) – one past the index of the last row in the subset. If missing, set to the end of the matrix.

Returns

sparse_batch – a sparse batch consisting of rows [rbegin, rend)

Return type

Batch

classmethod from_npy2d(mat, rbegin=0, rend=None, missing=None)

Get a dense batch from a 2D numpy matrix. If mat does not have order='C' (also known as row-major) or is not contiguous, a temporary copy will be made. If mat does not have dtype=numpy.float32, a temporary copy will be made also. Thus, as many as two temporary copies of data can be made. One should set input layout and type judiciously to conserve memory.

Parameters

  • mat (object of type numpy.ndarray, with dimension 2) – data matrix

  • rbegin (int, optional) – the index of the first row in the subset

  • rend (int, optional) – one past the index of the last row in the subset. If missing, set to the end of the matrix.

  • missing (float, optional) – value indicating missing value. If missing, set to numpy.nan.

Returns

dense_batch – a dense batch consisting of rows [rbegin, rend)

Return type

Batch

shape()

Get dimensions of the batch

Returns

dims – (number of rows, number of columns)

Return type

tuple of length 2

Treelite C API

Treelite exposes a set of C functions to enable interfacing with a variety of languages. This page will be most useful for:

  • those writing a new language binding (glue code).

  • those wanting to incorporate functions of treelite into their own native libraries.

We recommend the Python API for everyday uses.

Note

Use of C and C++ in treelite

Core logic of treelite are written in C++ to take advantage of higher abstractions. We provide C only interface here, as many more programming languages bind with C than with C++. See this page for more details.

Data matrix interface

Use the following functions to load and manipulate data from a variety of sources.

int TreeliteDMatrixCreateFromFile(const char *path, const char *format, int nthread, int verbose, DMatrixHandle *out)

create DMatrix from a file

Return

0 for success, -1 for failure

Parameters

  • path: file path

  • format: file format

  • nthread: number of threads to use

  • verbose: whether to produce extra messages

  • out: the created DMatrix

int TreeliteDMatrixCreateFromCSR(const float *data, const unsigned *col_ind, const size_t *row_ptr, size_t num_row, size_t num_col, DMatrixHandle *out)

create DMatrix from a (in-memory) CSR matrix

Return

0 for success, -1 for failure

Parameters

  • data: feature values

  • col_ind: feature indices

  • row_ptr: pointer to row headers

  • num_row: number of rows

  • num_col: number of columns

  • out: the created DMatrix

int TreeliteDMatrixCreateFromMat(const float *data, size_t num_row, size_t num_col, float missing_value, DMatrixHandle *out)

create DMatrix from a (in-memory) dense matrix

Return

0 for success, -1 for failure

Parameters

  • data: feature values

  • num_row: number of rows

  • num_col: number of columns

  • missing_value: value to represent missing value

  • out: the created DMatrix

int TreeliteDMatrixGetDimension(DMatrixHandle handle, size_t *out_num_row, size_t *out_num_col, size_t *out_nelem)

get dimensions of a DMatrix

Return

0 for success, -1 for failure

Parameters

  • handle: handle to DMatrix

  • out_num_row: used to set number of rows

  • out_num_col: used to set number of columns

  • out_nelem: used to set number of nonzero entries

int TreeliteDMatrixGetPreview(DMatrixHandle handle, const char **out_preview)

produce a human-readable preview of a DMatrix Will print first and last 25 non-zero entries, along with their locations

Return

0 for success, -1 for failure

Parameters

  • handle: handle to DMatrix

  • out_preview: used to save the address of the string literal

int TreeliteDMatrixGetArrays(DMatrixHandle handle, const float **out_data, const uint32_t **out_col_ind, const size_t **out_row_ptr)

extract three arrays (data, col_ind, row_ptr) that define a DMatrix.

Return

0 for success, -1 for failure

Parameters

  • handle: handle to DMatrix

  • out_data: used to save pointer to array containing feature values

  • out_col_ind: used to save pointer to array containing feature indices

  • out_row_ptr: used to save pointer to array containing pointers to row headers

int TreeliteDMatrixFree(DMatrixHandle handle)

delete DMatrix from memory

Return

0 for success, -1 for failure

Parameters

  • handle: handle to DMatrix

Branch annotator interface

Use the following functions to annotate branches in decision trees.

int TreeliteAnnotateBranch(ModelHandle model, DMatrixHandle dmat, int nthread, int verbose, AnnotationHandle *out)

annotate branches in a given model using frequency patterns in the training data.

Return

0 for success, -1 for failure

Parameters

  • model: model to annotate

  • dmat: training data matrix

  • nthread: number of threads to use

  • verbose: whether to produce extra messages

  • out: used to save handle for the created annotation

int TreeliteAnnotationSave(AnnotationHandle handle, const char *path)

save branch annotation to a JSON file

Return

0 for success, -1 for failure

Parameters

  • handle: annotation to save

  • path: path to JSON file

int TreeliteAnnotationFree(AnnotationHandle handle)

delete branch annotation from memory

Return

0 for success, -1 for failure

Parameters

  • handle: annotation to remove

Compiler interface

Use the following functions to produce optimize prediction subroutine (in C) from a given decision tree ensemble.

int TreeliteCompilerCreate(const char *name, CompilerHandle *out)

create a compiler with a given name

Return

0 for success, -1 for failure

Parameters

  • name: name of compiler

  • out: created compiler

int TreeliteCompilerSetParam(CompilerHandle handle, const char *name, const char *value)

set a parameter for a compiler

Return

0 for success, -1 for failure

Parameters

  • handle: compiler

  • name: name of parameter

  • value: value of parameter

int TreeliteCompilerGenerateCode(CompilerHandle compiler, ModelHandle model, int verbose, const char *dirpath)

generate prediction code from a tree ensemble model. The code will be C99 compliant. One header file (.h) will be generated, along with one or more source files (.c).

Usage example: 

TreeliteCompilerGenerateCode(compiler, model, 1, "./my/model");
// files to generate: ./my/model/header.h, ./my/model/main.c
// if parallel compilation is enabled:
// ./my/model/header.h, ./my/model/main.c, ./my/model/tu0.c,
// ./my/model/tu1.c, and so forth
Return

0 for success, -1 for failure

Parameters

  • compiler: handle for compiler

  • model: handle for tree ensemble model

  • verbose: whether to produce extra messages

  • dirpath: directory to store header and source files

int TreeliteCompilerFree(CompilerHandle handle)

delete compiler from memory

Return

0 for success, -1 for failure

Parameters

  • handle: compiler to remove

Model loader interface

Use the following functions to load decision tree ensemble models from a file. Treelite supports multiple model file formats.

int TreeliteLoadLightGBMModel(const char *filename, ModelHandle *out)

load a model file generated by LightGBM (Microsoft/LightGBM). The model file must contain a decision tree ensemble.

Return

0 for success, -1 for failure

Parameters

  • filename: name of model file

  • out: loaded model

int TreeliteLoadXGBoostModel(const char *filename, ModelHandle *out)

load a model file generated by XGBoost (dmlc/xgboost). The model file must contain a decision tree ensemble.

Return

0 for success, -1 for failure

Parameters

  • filename: name of model file

  • out: loaded model

int TreeliteLoadXGBoostModelFromMemoryBuffer(const void *buf, size_t len, ModelHandle *out)

load an XGBoost model from a memory buffer.

Return

0 for success, -1 for failure

Parameters

  • buf: memory buffer

  • len: size of memory buffer

  • out: loaded model

int TreeliteLoadProtobufModel(const char *filename, ModelHandle *out)

load a model in Protocol Buffers format. Protocol Buffers (google/protobuf) is a language- and platform-neutral mechanism for serializing structured data. See tree.proto for format spec.

Return

0 for success, -1 for failure

Parameters

  • filename: name of model file

  • out: loaded model

int TreeliteExportProtobufModel(const char *filename, ModelHandle model)

export a model in Protocol Buffers format. Protocol Buffers (google/protobuf) is a language- and platform-neutral mechanism for serializing structured data. See src/tree.proto for format spec.

Return

0 for success, -1 for failure

Parameters

  • filename: name of model file

  • model: model to export

int TreeliteQueryNumTree(ModelHandle handle, size_t *out)

Query the number of trees in the model.

Return

0 for success, -1 for failure

Parameters

  • handle: model to query

  • out: number of trees

int TreeliteQueryNumFeature(ModelHandle handle, size_t *out)

Query the number of features used in the model.

Return

0 for success, -1 for failure

Parameters

  • handle: model to query

  • out: number of features

int TreeliteQueryNumOutputGroups(ModelHandle handle, size_t *out)

Query the number of output groups of the model.

Return

0 for success, -1 for failure

Parameters

  • handle: model to query

  • out: number of output groups

int TreeliteSetTreeLimit(ModelHandle handle, size_t limit)

keep first N trees of model, limit must smaller than number of trees.

Return

0 for success, -1 for failure

Parameters

  • handle: model

  • limit: number of trees to keep

int TreeliteFreeModel(ModelHandle handle)

delete model from memory

Return

0 for success, -1 for failure

Parameters

  • handle: model to remove

Model builder interface

Use the following functions to incrementally build decisio n tree ensemble models.

int TreeliteCreateTreeBuilder(TreeBuilderHandle *out)

Create a new tree builder.

Return

0 for success; -1 for failure

Parameters

  • out: newly created tree builder

int TreeliteDeleteTreeBuilder(TreeBuilderHandle handle)

Delete a tree builder from memory.

Return

0 for success; -1 for failure

Parameters

  • handle: tree builder to remove

int TreeliteTreeBuilderCreateNode(TreeBuilderHandle handle, int node_key)

Create an empty node within a tree.

Return

0 for success; -1 for failure

Parameters

  • handle: tree builder

  • node_key: unique integer key to identify the new node

int TreeliteTreeBuilderDeleteNode(TreeBuilderHandle handle, int node_key)

Remove a node from a tree.

Return

0 for success; -1 for failure

Parameters

  • handle: tree builder

  • node_key: unique integer key to identify the node to be removed

int TreeliteTreeBuilderSetRootNode(TreeBuilderHandle handle, int node_key)

Set a node as the root of a tree.

Return

0 for success; -1 for failure

Parameters

  • handle: tree builder

  • node_key: unique integer key to identify the root node

int TreeliteTreeBuilderSetNumericalTestNode(TreeBuilderHandle handle, int node_key, unsigned feature_id, const char *opname, double threshold, int default_left, int left_child_key, int right_child_key)

Turn an empty node into a test node with numerical split. The test is in the form [feature value] OP [threshold]. Depending on the result of the test, either left or right child would be taken.

Return

0 for success; -1 for failure

Parameters

  • handle: tree builder

  • node_key: unique integer key to identify the node being modified; this node needs to be empty

  • feature_id: id of feature

  • opname: binary operator to use in the test

  • threshold: threshold value

  • default_left: default direction for missing values

  • left_child_key: unique integer key to identify the left child node

  • right_child_key: unique integer key to identify the right child node

int TreeliteTreeBuilderSetCategoricalTestNode(TreeBuilderHandle handle, int node_key, unsigned feature_id, const unsigned int *left_categories, size_t left_categories_len, int default_left, int left_child_key, int right_child_key)

Turn an empty node into a test node with categorical split. A list defines all categories that would be classified as the left side. Categories are integers ranging from 0 to (n-1), where n is the number of categories in that particular feature. Let’s assume n <= 64.

Return

0 for success; -1 for failure

Parameters

  • handle: tree builder

  • node_key: unique integer key to identify the node being modified; this node needs to be empty

  • feature_id: id of feature

  • left_categories: list of categories belonging to the left child

  • left_categories_len: length of left_cateogries

  • default_left: default direction for missing values

  • left_child_key: unique integer key to identify the left child node

  • right_child_key: unique integer key to identify the right child node

int TreeliteTreeBuilderSetLeafNode(TreeBuilderHandle handle, int node_key, double leaf_value)

Turn an empty node into a leaf node.

Return

0 for success; -1 for failure

Parameters

  • handle: tree builder

  • node_key: unique integer key to identify the node being modified; this node needs to be empty

  • leaf_value: leaf value (weight) of the leaf node

int TreeliteTreeBuilderSetLeafVectorNode(TreeBuilderHandle handle, int node_key, const double *leaf_vector, size_t leaf_vector_len)

Turn an empty node into a leaf vector node The leaf vector (collection of multiple leaf weights per leaf node) is useful for multi-class random forest classifier.

Return

0 for success; -1 for failure

Parameters

  • handle: tree builder

  • node_key: unique integer key to identify the node being modified; this node needs to be empty

  • leaf_vector: leaf vector of the leaf node

  • leaf_vector_len: length of leaf_vector

int TreeliteCreateModelBuilder(int num_feature, int num_output_group, int random_forest_flag, ModelBuilderHandle *out)

Create a new model builder.

Return

0 for success; -1 for failure

Parameters

  • num_feature: number of features used in model being built. We assume that all feature indices are between 0 and (num_feature - 1).

  • num_output_group: number of output groups. Set to 1 for binary classification and regression; >1 for multiclass classification

  • random_forest_flag: whether the model is a random forest. Set to 0 if the model is gradient boosted trees. Any nonzero value shall indicate that the model is a random forest.

  • out: newly created model builder

int TreeliteModelBuilderSetModelParam(ModelBuilderHandle handle, const char *name, const char *value)

Set a model parameter.

Return

0 for success; -1 for failure

Parameters

  • handle: model builder

  • name: name of parameter

  • value: value of parameter

int TreeliteDeleteModelBuilder(ModelBuilderHandle handle)

Delete a model builder from memory.

Return

0 for success; -1 for failure

Parameters

  • handle: model builder to remove

int TreeliteModelBuilderInsertTree(ModelBuilderHandle handle, TreeBuilderHandle tree_builder, int index)

Insert a tree at specified location.

Return

index of the new tree within the ensemble; -1 for failure

Parameters

  • handle: model builder

  • tree_builder: builder for the tree to be inserted. The tree must not be part of any other existing tree ensemble. Note: The tree_builder argument will become unusuable after the tree insertion. Should you want to modify the tree afterwards, use GetTree(*) method to get a fresh handle to the tree.

  • index: index of the element before which to insert the tree; use -1 to insert at the end

int TreeliteModelBuilderGetTree(ModelBuilderHandle handle, int index, TreeBuilderHandle *out)

Get a reference to a tree in the ensemble.

Return

0 for success; -1 for failure

Parameters

  • handle: model builder

  • index: index of the tree in the ensemble

  • out: used to save reference to the tree

int TreeliteModelBuilderDeleteTree(ModelBuilderHandle handle, int index)

Remove a tree from the ensemble.

Return

0 for success; -1 for failure

Parameters

  • handle: model builder

  • index: index of the tree that would be removed

int TreeliteModelBuilderCommitModel(ModelBuilderHandle handle, ModelHandle *out)

finalize the model and produce the in-memory representation

Return

0 for success; -1 for failure

Parameters

  • handle: model builder

  • out: used to save handle to in-memory representation of the finished model

Predictor interface

Use the following functions to load compiled prediction subroutines from shared libraries and to make predictions.

int TreeliteAssembleSparseBatch(const float *data, const uint32_t *col_ind, const size_t *row_ptr, size_t num_row, size_t num_col, CSRBatchHandle *out)

assemble a sparse batch

Return

0 for success, -1 for failure

Parameters

  • data: feature values

  • col_ind: feature indices

  • row_ptr: pointer to row headers

  • num_row: number of data rows in the batch

  • num_col: number of columns (features) in the batch

  • out: handle to sparse batch

int TreeliteDeleteSparseBatch(CSRBatchHandle handle)

delete a sparse batch from memory

Return

0 for success, -1 for failure

Parameters

  • handle: sparse batch

int TreeliteAssembleDenseBatch(const float *data, float missing_value, size_t num_row, size_t num_col, DenseBatchHandle *out)

assemble a dense batch

Return

0 for success, -1 for failure

Parameters

  • data: feature values

  • missing_value: value to represent the missing value

  • num_row: number of data rows in the batch

  • num_col: number of columns (features) in the batch

  • out: handle to sparse batch

int TreeliteDeleteDenseBatch(DenseBatchHandle handle)

delete a dense batch from memory

Return

0 for success, -1 for failure

Parameters

  • handle: dense batch

int TreeliteBatchGetDimension(void *handle, int batch_sparse, size_t *out_num_row, size_t *out_num_col)

get dimensions of a batch

Return

0 for success, -1 for failure

Parameters

  • handle: a batch of rows (must be of type SparseBatch or DenseBatch)

  • batch_sparse: whether the batch is sparse (true) or dense (false)

  • out_num_row: used to set number of rows

  • out_num_col: used to set number of columns

int TreelitePredictorLoad(const char *library_path, int num_worker_thread, PredictorHandle *out)

load prediction code into memory. This function assumes that the prediction code has been already compiled into a dynamic shared library object (.so/.dll/.dylib).

Return

0 for success, -1 for failure

Parameters

  • library_path: path to library object file containing prediction code

  • num_worker_thread: number of worker threads (-1 to use max number)

  • out: handle to predictor

int TreelitePredictorPredictBatch(PredictorHandle handle, void *batch, int batch_sparse, int verbose, int pred_margin, float *out_result, size_t *out_result_size)

Make predictions on a batch of data rows (synchronously). This function internally divides the workload among all worker threads.

Return

0 for success, -1 for failure

Parameters

  • handle: predictor

  • batch: a batch of rows (must be of type SparseBatch or DenseBatch)

  • batch_sparse: whether batch is sparse (1) or dense (0)

  • verbose: whether to produce extra messages

  • pred_margin: whether to produce raw margin scores instead of transformed probabilities

  • out_result: resulting output vector; use TreelitePredictorQueryResultSize() to allocate sufficient space

  • out_result_size: used to save length of the output vector, which is guaranteed to be less than or equal to TreelitePredictorQueryResultSize()

int TreelitePredictorPredictInst(PredictorHandle handle, union TreelitePredictorEntry *inst, int pred_margin, float *out_result, size_t *out_result_size)

Make predictions on a single data row (synchronously). The work will be scheduled to the calling thread.

Return

0 for success, -1 for failure

Parameters

int TreelitePredictorQueryResultSize(PredictorHandle handle, void *batch, int batch_sparse, size_t *out)

Given a batch of data rows, query the necessary size of array to hold predictions for all data points.

Return

0 for success, -1 for failure

Parameters

  • handle: predictor

  • batch: a batch of rows (must be of type SparseBatch or DenseBatch)

  • batch_sparse: whether batch is sparse (1) or dense (0)

  • out: used to store the length of prediction array

int TreelitePredictorQueryResultSizeSingleInst(PredictorHandle handle, size_t *out)

Query the necessary size of array to hold the prediction for a single data row.

Return

0 for success, -1 for failure

Parameters

  • handle: predictor

  • out: used to store the length of prediction array

int TreelitePredictorQueryNumOutputGroup(PredictorHandle handle, size_t *out)

Get the number of output groups in the loaded model The number is 1 for most tasks; it is greater than 1 for multiclass classifcation.

Return

0 for success, -1 for failure

Parameters

  • handle: predictor

  • out: length of prediction array

int TreelitePredictorQueryNumFeature(PredictorHandle handle, size_t *out)

Get the width (number of features) of each instance used to train the loaded model.

Return

0 for success, -1 for failure

Parameters

  • handle: predictor

  • out: number of features

int TreelitePredictorQueryPredTransform(PredictorHandle handle, char **out)

Get name of post prediction transformation used to train the loaded model.

Return

0 for success, -1 for failure

Parameters

  • handle: predictor

  • out: name of post prediction transformation

int TreelitePredictorQuerySigmoidAlpha(PredictorHandle handle, float *out)

Get alpha value of sigmoid transformation used to train the loaded model.

Return

0 for success, -1 for failure

Parameters

  • handle: predictor

  • out: alpha value of sigmoid transformation

int TreelitePredictorQueryGlobalBias(PredictorHandle handle, float *out)

Get global bias which adjusting predicted margin scores.

Return

0 for success, -1 for failure

Parameters

  • handle: predictor

  • out: global bias value

int TreelitePredictorFree(PredictorHandle handle)

delete predictor from memory

Return

0 for success, -1 for failure

Parameters

  • handle: predictor to remove

Handle types

Treelite uses C++ classes to define its internal data structures. In order to pass C++ objects to C functions, opaque handles are used. Opaque handles are void* pointers that store raw memory addresses.

typedef void *DMatrixHandle

handle to a data matrix

typedef void *ModelHandle

handle to a decision tree ensemble model

typedef void *TreeBuilderHandle

handle to tree builder class

typedef void *ModelBuilderHandle

handle to ensemble builder class

typedef void *AnnotationHandle

handle to branch annotation data

typedef void *CompilerHandle

handle to compiler class

typedef void *PredictorHandle

handle to predictor class

typedef void *CSRBatchHandle

handle to batch of sparse data rows

typedef void *DenseBatchHandle

handle to batch of dense data rows

Treelite Java API (EXPERIMENTAL)

ml.dmlc.treelite4j.java

BatchBuilder
public class BatchBuilder

Collection of utility functions to create batch objects

Author

Philip Cho

Methods
CreateDenseBatch
public static DenseBatch CreateDenseBatch(Iterator<DataPoint> dIter)

Assemble a dense batch from a list of data points

Parameters

  • dIter – Iterator of data points

Throws
Returns

Created dense batch

CreateSparseBatch
public static SparseBatch CreateSparseBatch(Iterator<DataPoint> dIter)

Assemble a sparse batch from a list of data points

Parameters

  • dIter – Iterator of data points

Throws
Returns

Created sparse batch

LoadDatasetFromLibSVM
public static List<DataPoint> LoadDatasetFromLibSVM(String filename)

Load a LIBSVM data file and construct a list of data points

Parameters

  • filename – path to LIBSVM file

Throws
Returns

Created list of data points

Data
public interface Data

Interface to specify a single data entry.

Author

Philip Cho

Methods
getFValue
public float getFValue()

Obtain the floating-point value stored by the entry.

Returns

floating-point value

getQValue
public int getQValue()

Obtain the integer value stored by the entry.

Returns

integer value

isMissing
public boolean isMissing()

Test whether the entry is missing.

Returns

whether the entry is missing

setFValue
public void setFValue(float val)

Assign a floating-point value to the entry.

Parameters

  • val – value to set

setMissing
public void setMissing()

Designate the entry as missing.

setQValue
public void setQValue(int val)

Assign an integer value to the entry. This is useful when feature values and split thresholds are quantized into integers.

Parameters

  • val – value to set

write
public void write(OutputStream out)

Serialize

Parameters

  • out –

Throws

  • IOException –

DenseBatch
public class DenseBatch

2D dense batch, laid out in row-major layout

Author

Philip Cho

Constructors
DenseBatch
public DenseBatch(float[] data, float missing_value, int num_row, int num_col)

Create a dense batch representing a 2D dense matrix

Parameters

  • data – array of entries, should be of length [num_row]*[num_col]

  • missing_value – floating-point value representing a missing value; usually set of Float.NaN.

  • num_row – number of rows (data instances) in the matrix

  • num_row – number of columns (features) in the matrix

Throws

  • TreeliteError –

Return

Created dense batch

Methods
dispose
public synchronized void dispose()

Destructor, to be called when the object is garbage collected

finalize
protected void finalize()
getHandle
public long getHandle()

Get the underlying native handle

Returns

Integer representing memory address

DenseBatchTest
public class DenseBatchTest

Test cases for dense batch

Author

Philip Cho

Methods
testDenseBatchBasic
public void testDenseBatchBasic()
Entry
public class Entry extends Union implements Data

A reference implementation for the Data interface. This class is used to specify a single data entry. This implementation uses a C-style union so as to save memory space.

Author

Philip Cho

Fields
fvalue
public Float32 fvalue

The value of the entry, in floating-point representation

missing
public Signed32 missing

If missing == -1, the entry is considered missing. Otherwise, the entry is not missing; check the other fields to obtain the actual value of the entry.

qvalue
public Signed32 qvalue

The value of the entry, in integer representation

Methods
byteOrder
public ByteOrder byteOrder()

The byte-order to use when serializing.

Returns

ByteOrder.LITTLE_ENDIAN, since this class will be serialized using the litte-endian byte order.

getFValue
public float getFValue()

Obtain the floating-point value stored by the entry.

Returns

floating-point value

getQValue
public int getQValue()

Obtain the integer value stored by the entry.

Returns

integer value

isMissing
public boolean isMissing()

Test whether the entry is missing.

Returns

whether the entry is missing

setFValue
public void setFValue(float val)

Assign a floating-point value to the entry.

Parameters

  • val – value to set

setMissing
public void setMissing()

Designate the entry as missing.

setQValue
public void setQValue(int val)

Assign an integer value to the entry. This is useful when feature values and split thresholds are quantized into integers.

Parameters

  • val – value to set

InferenceEngine
public interface InferenceEngine

Interface for making inference (WORK-IN-PROGRESS)

Author

James Liu

Methods
getNumFeature
public int getNumFeature()

Get the number of features used by the compiled model. Call this method to allocate array for storing data entries of a single instance.

Returns

Number of features

getNumOutputGroup
public int getNumOutputGroup()

Get the number of output groups for the compiled model. This number is 1 for tasks other than multi-class classification. For multi-class classification task, the number is equal to the number of classes.

Returns

Number of output groups

predict
public float[] predict(Data[] inst, boolean pred_margin)

Perform single-instance prediction

Parameters

  • inst – array of data entires(features) comprising the instance

  • pred_margin – whether to predict a probability or a raw margin score

Returns

Resulting predictions, of dimension [num_output_group]

NativeLibLoader
public class NativeLibLoader

Class to load the native lib libtreelite4j.dylib. Normally, users do not have to worry about this class, since mvn package automatically bundles libtreelite4j.dylib into the JAR file. However, the method createTempFileFromResource may be useful to some users, who would like to bundle other files into the JAR file as well.

Author

Philip Cho

Methods
createTempFileFromResource
public static String createTempFileFromResource(String path)

Create a temp file that copies the resource from current JAR archive

The file from JAR is copied into system temp file. The temporary file is deleted after exiting. Method uses String as filename because the pathname is “abstract”, not system-dependent.

The restrictions of File.createTempFile(java.lang.String,java.lang.String) apply to path.

Parameters

  • path – Path to the resources in the jar

Throws

  • IOException – When the temp file could not be created

  • IllegalArgumentException – When the file name contains invalid letters

Returns

The created temp file.

initTreeliteRuntime
static synchronized void initTreeliteRuntime()

Initialization method to load the native treelite4j lib at startup

Throws

  • IOException – when treelite4j lib is not found

Predictor
public class Predictor implements Serializable

Treelite Predictor

Author

Philip Cho

Constructors
Predictor
public Predictor(String libpath, int nthread, boolean verbose)

Create a Predictor by loading a shared library (dll/so/dylib). The library is expected to contain the compiled code for making prediction for a particular tree ensemble model. The predictor also spawns a fixed-size pool of worker threads, who will wait for prediction tasks. (Note that the worker threads will go to sleep when no prediction task is available, to free up CPU cycles for other processes.)

Parameters

  • libpath – Path to the shared library

  • nthread – Number of workers threads to spawn. Set to -1 to use default, i.e., to launch as many threads as CPU cores available on the system. You are not allowed to launch more threads than CPU cores. Setting nthread=1 indicates that the main thread should be exclusively used.

  • verbose – Whether to print extra diagnostic messages

Throws

  • TreeliteError –

Return

Created Predictor

Methods
GetGlobalBias
public float GetGlobalBias()

Get global bias which adjusting predicted margin scores.

Returns

Value of global bias

GetNumFeature
public int GetNumFeature()

Get the number of features used by the compiled model. Call this method to allocate array for storing data entries of a single instance.

Returns

Number of features

GetNumOutputGroup
public int GetNumOutputGroup()

Get the number of output groups for the compiled model. This number is 1 for tasks other than multi-class classification. For multi-class classification task, the number is equal to the number of classes.

Returns

Number of output groups

GetPredTransform
public String GetPredTransform()

Get name of post prediction transformation used to train the loaded model.

Returns

Name of (post-)prediction transformation

GetSigmoidAlpha
public float GetSigmoidAlpha()

Get alpha value in sigmoid transformation used to train the loaded model.

Returns

Alpha value of sigmoid transformation

dispose
public synchronized void dispose()

Destructor, to be called when the object is garbage collected

finalize
protected void finalize()
predict
public float[] predict(Data[] inst, boolean pred_margin)

Perform single-instance prediction. Prediction is run by the calling thread.

Parameters

  • inst – array of data entires comprising the instance

  • pred_margin – whether to predict a probability or a raw margin score

Returns

Resulting predictions, of dimension [num_output_group]

predict
public float[][] predict(SparseBatch batch, boolean verbose, boolean pred_margin)

Perform batch prediction with a 2D sparse data matrix. Worker threads will internally divide up work for batch prediction. Note that this function will be blocked by mutex when worker_thread > 1. In order to use multiple threads to process multiple prediction requests simultaneously, use Predictor.predict(Data[], boolean) instead or keep worker_thread = 1.

Parameters

  • batch – a SparseBatch, representing a slice of a 2D sparse matrix

  • verbose – whether to print extra diagnostic messages

  • pred_margin – whether to predict probabilities or raw margin scores

Returns

Resulting predictions, of dimension [num_row]*[num_output_group]

predict
public float[][] predict(DenseBatch batch, boolean verbose, boolean pred_margin)

Perform batch prediction with a 2D dense data matrix. Worker threads will internally divide up work for batch prediction. Note that this function will be blocked by mutex when worker_thread > 1. In order to use multiple threads to process multiple prediction requests simultaneously, use Predictor.predict(Data[], boolean) instead or keep worker_thread = 1.

Parameters

  • batch – a DenseBatch, representing a slice of a 2D dense matrix

  • verbose – whether to print extra diagnostic messages

  • pred_margin – whether to predict probabilities or raw margin scores

Returns

Resulting predictions, of dimension [num_row]*[num_output_group]

PredictorTest
public class PredictorTest

Test cases for Treelite Predictor

Author

Philip Cho

Constructors
PredictorTest
public PredictorTest()
Methods
LoadArrayFromText
public static float[] LoadArrayFromText(String filename)
testPredict
public void testPredict()
testPredictInst
public void testPredictInst()
testPredictMargin
public void testPredictMargin()
testPredictorBasic
public void testPredictorBasic()
testSerialization
public void testSerialization()
SparseBatch
public class SparseBatch

2D sparse batch, laid out in CSR (Compressed Sparse Row) layout

Author

Philip Cho

Constructors
SparseBatch
public SparseBatch(float[] data, int[] col_ind, long[] row_ptr, int num_row, int num_col)

Create a sparse batch representing a 2D sparse matrix

Parameters

  • data – nonzero (non-missing) entries

  • col_ind – corresponding column indices, should be of same length as data

  • row_ptr – offsets to define each instance, should be of length [num_row]+1

  • num_row – number of rows (data instances) in the matrix

  • num_row – number of columns (features) in the matrix

Throws

  • TreeliteError –

Return

Created sparse batch

Methods
dispose
public synchronized void dispose()

Destructor, to be called when the object is garbage collected

finalize
protected void finalize()
getHandle
public long getHandle()

Get the underlying native handle

Returns

Integer representing memory address

SparseBatchTest
public class SparseBatchTest

Test cases for sparse batch

Author

Philip Cho

Methods
testSparseBatchBasic
public void testSparseBatchBasic()
testSparseBatchBuilder
public void testSparseBatchBuilder()
TreeliteError
public class TreeliteError extends Exception

Custom error class for Treelite

Author

Philip Cho

Constructors
TreeliteError
public TreeliteError(String message)
TreeliteJNI
class TreeliteJNI

Treelite prediction runtime JNI functions

Author

Philip Cho

Methods
TreeliteAssembleDenseBatch
public static final native int TreeliteAssembleDenseBatch(float[] data, float missing_value, long num_row, long num_col, long[] out)
TreeliteAssembleSparseBatch
public static final native int TreeliteAssembleSparseBatch(float[] data, int[] col_ind, long[] row_ptr, long num_row, long num_col, long[] out)
TreeliteBatchGetDimension
public static final native int TreeliteBatchGetDimension(long handle, boolean batch_sparse, long[] out_num_row, long[] out_num_col)
TreeliteDeleteDenseBatch
public static final native int TreeliteDeleteDenseBatch(long handle, float[] data)
TreeliteDeleteSparseBatch
public static final native int TreeliteDeleteSparseBatch(long handle, float[] data, int[] col_ind, long[] row_ptr)
TreeliteGetLastError
public static final native String TreeliteGetLastError()
TreelitePredictorFree
public static final native int TreelitePredictorFree(long handle)
TreelitePredictorLoad
public static final native int TreelitePredictorLoad(String library_path, int num_worker_thread, long[] out)
TreelitePredictorPredictBatch
public static final native int TreelitePredictorPredictBatch(long handle, long batch, boolean batch_sparse, boolean verbose, boolean pred_margin, float[] out_result, long[] out_result_size)
TreelitePredictorPredictInst
public static final native int TreelitePredictorPredictInst(long handle, byte[] inst, boolean pred_margin, float[] out_result, long[] out_result_size)
TreelitePredictorQueryGlobalBias
public static final native int TreelitePredictorQueryGlobalBias(long handle, float[] out)
TreelitePredictorQueryNumFeature
public static final native int TreelitePredictorQueryNumFeature(long handle, long[] out)
TreelitePredictorQueryNumOutputGroup
public static final native int TreelitePredictorQueryNumOutputGroup(long handle, long[] out)
TreelitePredictorQueryPredTransform
public static final native int TreelitePredictorQueryPredTransform(long handle, String[] out)
TreelitePredictorQueryResultSize
public static final native int TreelitePredictorQueryResultSize(long handle, long batch, boolean batch_sparse, long[] out)
TreelitePredictorQueryResultSizeSingleInst
public static final native int TreelitePredictorQueryResultSizeSingleInst(long handle, long[] out)
TreelitePredictorQuerySigmoidAlpha
public static final native int TreelitePredictorQuerySigmoidAlpha(long handle, float[] out)
checkCall
static void checkCall(int ret)

Check the return code of the JNI call.

Throws

Knobs and Parameters

Compiler Parameters

Compiler parameters influence the way the prediction subroutine is generated from a tree ensemble model.

std::string annotate_in

name of model annotation file. Use the class treelite.Annotator to generate this file.

int quantize

whether to quantize threshold points (0: no, >0: yes)

int parallel_comp

option to enable parallel compilation; if set to nonzero, the trees will be evely distributed into [parallel_comp] files. Set this option to improve compilation time and reduce memory consumption during compilation.

int verbose

if >0, produce extra messages

std::string native_lib_name

native lib name (without extension)

double code_folding_req

parameter for folding rarely visited subtrees (no if/else blocks); all nodes whose data counts are lower than that of the root node of the decision tree by [code_folding_req] will be folded. To diable folding, set to +inf. If hessian sums are available, they will be used as proxies of data counts.

int dump_array_as_elf

Only applicable when compiler is set to failsafe. If set to a positive value, the fail-safe compiler will not emit large constant arrays to the C code. Instead, the arrays will be emitted as an ELF binary (Linux only). For large arrays, it is much faster to directly dump ELF binaries than to pass them to a C compiler.

Model Parameters

Model parameters are used by ModelBuilder class to clarify certain behaviors of a tree ensemble model.

std::string pred_transform

name of prediction transform function

This parameter specifies how to transform raw margin values into final predictions. By default, this is set to 'identity', which means no transformation.

For the multi-class classification task, pred_transfrom must be one of the following values: 

 - identity_multiclass
   do not transform. The output will be a matrix with dimensions
   [number of data points] * [number of classes] that contains the margin score
   for every (data point, class) pair.
 - max_index
   compute the most probable class for each data point and output the class
   index. The output will be a vector of length [number of data points] that
   contains the most likely class of each data point.
 - softmax
   use the softmax function to transform a multi-dimensional vector into a
   proper probability distribution. The output will be a matrix with dimensions
   [number of data points] * [number of classes] that contains the predicted
   probability of each data point belonging to each class.
 - multiclass_ova
   apply the sigmoid function element-wise to the margin matrix. The output will
   be a matrix with dimensions [number of data points] * [number of classes].
For all other tasks (e.g. regression, binary classification, ranking etc.), pred_transfrom must be one of the following values: 
  - identity
    do not transform. The output will be a vector of length
    [number of data points] that contains the margin score for every data point.
  - sigmoid
    apply the sigmoid function element-wise to the margin vector. The output
    will be a vector of length [number of data points] that contains the
    probability of each data point belonging to the positive class.
  - exponential
    apply the exponential function (exp) element-wise to the margin vector. The
    output will be a vector of length [number of data points].
  - logarithm_one_plus_exp
    apply the function f(x) = log(1 + exp(x)) element-wise to the margin vector.
    The output will be a vector of length [number of data points].

float sigmoid_alpha

scaling parameter for sigmoid function sigmoid(x) = 1 / (1 + exp(-alpha * x))

This parameter is used only when pred_transform is set to 'sigmoid'. It must be strictly positive; if unspecified, it is set to 1.0.

float global_bias

global bias of the model

Predicted margin scores of all instances will be adjusted by the global bias. If unspecified, the bias is set to zero.

© Copyright 2017, DMLC. All rights reserved.. Created using Sphinx.