Treelite : toolbox for decision tree deployment

Treelite is a flexible toolbox for efficient deployment of decision tree ensembles.

Star Watch

You are currently browsing the documentation of a stable version of treelite: 0.32.

Why treelite?

Compile and optimize your model for fast prediction

Treelite compiles your tree model into optimized shared library. A Benchmark demonstrates 2-6x improvement in prediction throughput, due to more efficient use of compute resources.

Use machine learning package of your choice

Treelite accommodates a wide range of decision tree ensemble models. In particular, it handles both random forests and gradient boosted trees.

Treelite can read models produced by XGBoost, LightGBM, and scikit-learn. In cases where you are using another package to train your model, you may use the flexible builder class.

Deploy with minimal dependencies

It is a great hassle to install machine learning packages (e.g. XGBoost, LightGBM, scikit-learn, etc.) on every machine your tree model will run. This is the case no longer: treelite will export your model as a stand-alone prediction library so that predictions will be made without any machine learning package installed.

Quick start

Install treelite from PyPI:

pip3 install --user treelite

Import your tree ensemble model into treelite:

import treelite
model = treelite.Model.load('my_model.model', model_format='xgboost')

Deploy a source archive:

# Produce a zipped source directory, containing all model information
# Run `make` on the target machine
model.export_srcpkg(platform='unix', toolchain='gcc',
                    pkgpath='./mymodel.zip', libname='mymodel.so',
                    verbose=True)

Deploy a shared library:

# Like export_srcpkg, but generates a shared library immediately
# Use this only when the host and target machines are compatible
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True)

Make predictions on the target machine:

import treelite.runtime
predictor = treelite.runtime.Predictor('./mymodel.so', verbose=True)
batch = treelite.runtime.Batch.from_npy2d(X)
out_pred = predictor.predict(batch)

Read First tutorial for a more detailed example. See Deploying models for additional instructions on deployment.

Note

A note on API compatibility

Since treelite is in early development, its API may change substantially in the future.

Benchmark

See the page Benchmark for details.

How treelite works

(Click to enlarge)

The workflow involves two distinct machines: the host machine that generates prediction subroutine from a given tree model, and the target machine that runs the subroutine. The two machines exchange a single C file that contains all relevant information about the tree model. Only the host machine needs to have treelite installed; the target machine requires only a working C compiler.

Contents

Installation

You may choose one of two methods to install treelite on your system:

• Download binary releases from PyPI (Recommended)
• Compile treelite from the source

Download binary releases from PyPI (Recommended)

This is probably the most convenient method. Simply type

pip3 install --user treelite

to install the treelite package. The command will locate the binary release that is compatible with your current platform. Check the installation by running

import treelite

in an interactive Python session. This method is available for only Windows, Mac OS X, and Linux. For other operating systems, see the next section.

Compile treelite from the source

Installation consists of two steps:

1. Build the shared libraries from C++ code (See the note below for the list.)
2. Install the Python package.

Note

List of libraries created

There will be two libraries created: the main library, for producing optimized prediction subroutines; and the runtime library, for deploying these subroutines in the wild for actual prediction tasks.

Operating System	Main library	Runtime library
Windows	treelite.dll	treelite_runtime.dll
Mac OS X	libtreelite.dylib	libtreelite_runtime.dylib
Linux / other UNIX	libtreelite.so	libtreelite_runtime.so

To get started, clone treelite repo from GitHub. It is important to clone the submodules with --recursive option.

git clone --recursive https://github.com/dmlc/treelite.git
cd treelite

The next step is to build the shared libraries.

1-1. Compiling shared libraries on Linux and Mac OS X

Here, we use CMake to generate a Makefile:

mkdir build
cd build
cmake ..

Once CMake finished running, simply invoke GNU Make to obtain the shared libraries.

make

The compiled libraries will be under the lib/ directory.

Note

Compiling treelite with multithreading on Mac OS X

The default clang installation on Mac OS X does not support OpenMP, the language construct for multithreading. To enable multithreading in treelite, we recommend that you install gcc 7.x using Homebrew:

brew install gcc@7

After g++ is installed, run CMake again with gcc as the C++ compiler:

cmake .. -DCMAKE_CXX_COMPILER=g++-7 -DCMAKE_C_COMPILER=gcc-7

1-2. Compiling shared libraries on Windows

We can use CMake to generate a Visual Studio project. The following snippet assumes that Visual Studio 2017 is installed. Adjust the version depending on the copy that’s installed on your system.

mkdir build
cd build
cmake .. -G"Visual Studio 15 2017 Win64"

Note

Visual Studio 2015 or newer is required

A large part of treelite has been written using the C++11 standard. Visual Studio 2015 is the first version that supports the new standard to fullest extent.

Once CMake finished running, open the generated solution file (treelite.sln) in Visual Studio. From the top menu, select Build > Build Solution. The compiled libraries will be under the lib/ directory.

2. Installing Python package

The Python package is located at the python subdirectory. There are several ways to install the package:

1. Install system-wide, which requires root permission

cd python
sudo python setup.py install

You will need Python setuptools module for this to work. It is often part of the core Python installation. Should it be necessary, the package can be installed using pip:

pip install -U pip setuptools

2. Install for only current user

This is useful if you do not have the administrative rights.

cd python
python setup.py develop --user

Note

Recompiling treelite

Every time the C++ portion of treelite gets re-compiled, the Python package must be re-installed for the new library to take effect.

3. Set the environment variable PYTHONPATH to locate treelite package

Only set the environment variable PYTHONPATH to tell Python where to find the treelite package. This is useful for developers, as any changes made to C++ code will be immediately visible to Python side without re-running setup.py.

export PYTHONPATH=path/to/treelite/python
python          # enter interactive session

Note

Compiling with Protocol Buffers support

If your system has Protocol Buffers (google/protobuf) library installed, treelite will be compiled with Protocol Buffers support. That is, you will able to read tree ensemble models that had been serialized using Protocol Buffers. Protocol Buffers support is strictly optional; treelite can be compiled without it. Should you decide to use Protocol Buffers, you should specify your ensemble model according to the specification src/tree.proto.

Binary releases hosted on PyPI have been compiled with Protocol Buffers support.

On Windows, you should specify the root directory containing Protobuf compilers and libraries by setting the environment variable CMAKE_PREFIX_PATH as follows:

mkdir build
cd build

:: Specify location of protobuf (Protocol Buffers)
set CMAKE_PREFIX_PATH=C:\path\to\protobuf
cmake .. -G"Visual Studio 15 2017 Win64"

Tutorials

This page lists tutorials about treelite.

First tutorial

This tutorial will demonstrate the basic workflow.

import treelite

Regression Example

In this tutorial, we will use a small regression example to describe the full workflow.

Load the Boston house prices dataset

Let us use the Boston house prices dataset from scikit-learn (sklearn.datasets.load_boston()). It consists of 506 houses with 13 distinct features:

from sklearn.datasets import load_boston
X, y = load_boston(return_X_y=True)
print('dimensions of X = {}'.format(X.shape))
print('dimensions of y = {}'.format(y.shape))

Train a tree ensemble model using XGBoost

The first step is to train a tree ensemble model using XGBoost (dmlc/xgboost).

Disclaimer: Treelite does NOT depend on the XGBoost package in any way. XGBoost was used here only to provide a working example.

import xgboost
dtrain = xgboost.DMatrix(X, label=y)
params = {'max_depth':3, 'eta':1, 'silent':1, 'objective':'reg:linear',
          'eval_metric':'rmse'}
bst = xgboost.train(params, dtrain, 20, [(dtrain, 'train')])

Pass XGBoost model into treelite

Next, we feed the trained model into treelite. If you used XGBoost to train the model, it takes only one line of code:

model = treelite.Model.from_xgboost(bst)

Note

Using other packages to train decision trees

With additional work, you can use models trained with other machine learning packages. See this page for instructions.

Generate shared library

Given a tree ensemble model, treelite will produce a prediction subroutine (internally represented as a C program). To use the subroutine for prediction task, we package it as a dynamic shared library, which exports the prediction subroutine for other programs to use.

Before proceeding, you should decide which of the following compilers is available on your system and set the variable toolchain appropriately:

• gcc
• clang
• msvc (Microsoft Visual C++)

toolchain = 'clang'   # change this value as necessary

The choice of toolchain will be used to compile the prediction subroutine into native code.

Now we are ready to generate the library.

model.export_lib(toolchain=toolchain, libpath='./mymodel.dylib', verbose=True)
                            #                            ^^^^^
                            # set correct file extension here; see the following paragraph

Note

File extension for shared library

Make sure to use the correct file extension for the library, depending on the operating system:

• Windows: .dll
• Mac OS X: .dylib
• Linux / Other UNIX: .so

Note

Want to deploy the model to another machine?

This tutorial assumes that predictions will be made on the same machine that is running treelite. If you’d like to deploy your model to another machine (that may not have treelite installed), see the page Deploying models.

Note

Reducing compilation time for large models

For large models, export_lib() may take a long time to finish. To reduce compilation time, enable the parallel_comp option by writing

model.export_lib(toolchain=toolchain, libpath='./mymodel.dylib',
                 params={'parallel_comp': 32}, verbose=True)

which splits the prediction subroutine into 32 source files that gets compiled in parallel. Adjust this number according to the number of cores on your machine.

Use the shared library to make predictions

Once the shared library has been generated, we feed it into a separate module (treelite.runtime) known as the runtime. The optimized prediction subroutine is exposed through the Predictor class:

import treelite.runtime     # runtime module
predictor = treelite.runtime.Predictor('./mymodel.dylib', verbose=True)

We decide on which of the houses in X we should make predictions for. Say, from 10th house to 20th:

batch = treelite.runtime.Batch.from_npy2d(X, rbegin=10, rend=20)

We used the method from_npy2d() because the matrix X was a dense NumPy array (numpy.ndarray). If X were a sparse matrix (scipy.sparse.csr_matrix), we would have used the method from_csr() instead.

out_pred = predictor.predict(batch)
print(out_pred)

Importing tree ensemble models

Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.

Importing XGBoost models

XGBoost (dmlc/xgboost) is a fast, scalable package for gradient boosting. Both treelite and XGBoost are hosted by the DMLC (Distributed Machine Learning Community) group.

Treelite plays well with XGBoost — if you used XGBoost to train your ensemble model, you need only one line of code to import it. Depending on where your model is located, you should do one of the following:

• Load XGBoost model from a xgboost.Booster object

# bst = an object of type xgboost.Booster
model = Model.from_xgboost(bst)

• Load XGBoost model from a binary model file

# model had been saved to a file named my_model.model
# notice the second argument format='xgboost'
model = Model.load('my_model.model', format='xgboost')

Importing LightGBM models

LightGBM (Microsoft/LightGBM) is another well known machine learning package for gradient boosting. To import models generated by LightGBM, use the load() method with argument format='lightgbm':

# model had been saved to a file named my_model.txt
# notice the second argument format='lightgbm'
model = Model.load('my_model.txt', format='lightgbm')

Importing scikit-learn models

Scikit-learn (scikit-learn/scikit-learn) is a Python machine learning package known for its versatility and ease of use. It supports a wide variety of models and algorithms. The following kinds of models can be imported into treelite.

• sklearn.ensemble.RandomForestRegressor
• sklearn.ensemble.RandomForestClassifier
• sklearn.ensemble.GradientBoostingRegressor
• sklearn.ensemble.GradientBoostingClassifier

To import scikit-learn models, use treelite.gallery.sklearn.import_model():

# clf is the model object generated by scikit-learn
import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)

How about other packages?

If you used other packages to train your ensemble model, you’d need to specify the model programmatically. There are two ways to do this:

• Specifying models using model builder
• Specifying models using Protocol Buffers

Optimizing prediction subroutine

Treelite offers system-level optimizations to boost prediction performance. Notice that the model information is wholly preserved; the optimizations only affect the manner at which prediction is performed.

Annotate conditional branches

This optimization analyzes and annotates every threshold conditions in the test nodes to improve performance.

How to use

The first step is to generate the branch annotation record for your ensemble model. Make sure to have your training data ready.

# model = your ensemble model (object of type treelite.Model)
# dmat = training data (object of type treelite.DMatrix)

# Create annotator object
annotator = treelite.Annotator()
# Annotate branches by iterating over the training data
annotator.annotate_branch(model=model, dmat=dmat, verbose=True)
# Save the branch annotation record as a JSON file
annotator.save(path='mymodel-annotation.json')

To utilize the branch annotation record, supply the compiler parameter annotate_in when exporting the model:

# Export a source directory
model.compile(dirpath='./mymodel', verbose=True,
              params={'quantize': 1})

# Export a source directory, packaged in a zip archive
model.export_srcpkg(platform='unix', toolchain='gcc', pkgpath='./mymodel.zip',
                    libname='mymodel.so', verbose=True,
                    params={'quantize': 1})

# Export a shared library
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True,
                 params={'quantize': 1})

Technical details

Rationale

Modern CPUs heavily rely on a technique known as branch prediction, in which they “guess” the result of the conditional expression in each if-else branch ahead of time. Given a program

if ( [conditional expression] ) {
  foo();
} else {
  bar();
}

the CPU will pre-fetch the instructions for the function foo() if the given condition is likely to be true. On the other hand, if the condition is likely to be false, the CPU will pre-fetch the instructions for the function bar(). It suffices to say that correctly predicting conditional branches has great impact on performance. Each time the CPU predicts a branch correctly, it can keep the instructions it had pre-fetched earlier. Each time the CPU fails to predict, it must throw away the pre-fetched instructions and fetch anew another set of instructions. If you’d like to learn more about the importance of branch prediction, read this excellent introductory article from Stanford.

The prediction subroutine for a decision tree ensemble is problematic, as it is replete with conditional branches that will have to be guessed well:

/* A slice of prediction subroutine */
float predict_margin(const float* data) {
  float sum = 0.0f;
  if (!(data[0].missing != -1) || data[0].fvalue <= 9.5) {
    if (!(data[0].missing != -1) || data[0].fvalue <= 3.5) {
      if (!(data[10].missing != -1) || data[10].fvalue <= 0.74185) {
        if (!(data[0].missing != -1) || data[0].fvalue <= 1.5) {
          if (!(data[2].missing != -1) || data[2].fvalue <= 2.08671) {
            if ( (data[4].missing != -1) && data[4].fvalue <= 2.02632) {
              if (!(data[3].missing != -1) || data[3].fvalue <= 0.763339) {
                sum += (float)0.00758165;
              } else {
                sum += (float)0.0060202;
              }
            } else {
              if ( (data[1].missing != -1) && data[1].fvalue <= 0.0397456) {
                sum += (float)0.00415399;
              } else {
                sum += (float)0.00821985;
              }
            }
/* and so forth... */

In fact, each threshold condition in the test nodes will need to be predicted. While CPUs lack adequate information to make good guesses on these conditions, we can help by providing that information.

Mechanism for supplying the C compiler with branch information

We predict the likelihood of each condition by counting the number of data points from the training data that satisfy that condition. See the diagram below for an illustration.

If a condition is true at least 50% of the time (over the training data), the condition is labeled as “expected to be true”:

/* expected to be true */
if ( __builtin_expect( [condition], 1 ) ) {
  ...
} else {
  ...
}

On the other hand, if a condition is false at least 50% of the time, the condition is labeled as “expected to be false”:

/* expected to be false */
if ( __builtin_expect( [condition], 0 ) ) {
  ...
} else {
  ...
}

Note

On the expression __builtin_expect

The __builtin_expect expression is a compiler intrinsic to supply the C compiler with branch prediction information. Both gcc and clang support it. Unfortunately, Microsoft Visual C++ does not. To take advantage of branch annotation, make sure to use gcc or clang on the target machine.

Use integer thresholds for conditions

This optimization replaces all thresholds in the test nodes with integers so that each threshold condition performs integer comparison instead of the usual floating-point comparison. The thresholds are said to be quantized into integer indices.

BEFORE:

if (data[3].fvalue < 1.5) {  /* floating-point comparison */
  ...
}

AFTER:

if (data[3].qvalue < 3) {     /* integer comparison */
  ...
}

How to use

Simply add the compiler parameter quantize=1 when exporting the model:

# Export a source directory
model.compile(dirpath='./mymodel', verbose=True,
              params={'quantize': 1})

# Export a source directory, packaged in a zip archive
model.export_srcpkg(platform='unix', toolchain='gcc', pkgpath='./mymodel.zip',
                    libname='mymodel.so', verbose=True,
                    params={'quantize': 1})

# Export a shared library
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True,
                 params={'quantize': 1})

Technical details

Rationale

On some platforms such as x86-64, replacing floating-point thresholds with integers helps improve performance by 1) reducing executable code size and 2) improving data locality. This is so because on these platforms, integer constants can be embedded as part of the comparison instruction, whereas floating-point constants cannot.

Let’s look at x86-64 platform. The integer comparison

a <= 4

produces one assembly instruction:

cmpl    $4, 8(%rsp)       ;    8(%rsp) contains the variable a

Since the integer constant 4 got embedded into the comparison instruction cmpl, we only had to fetch the variable a from memory.

On the other hand, the floating-point comparison

b < 1.2f

produces two assembly instructions:

movss   250(%rip), %xmm0  ;  250(%rip) contains the constant 1.2f
ucomiss  12(%rsp), %xmm0  ;   12(%rsp) contains the variable b

Notice that the floating-point constant 1.2f did not get embedded into the comparison instruction ucomiss. The constant had to be fetched (with movss) into the register xmm0 before the comparsion could take place. To summarize,

• a floating-point comparison takes twice as many instructions as an integer comparsion, increasing the executable code size;
• a floating-point comparison involves an extra fetch instruction (movss), potentially causing a cache miss.

Caveats. As we’ll see in the next section, using integer thresholds will add overhead costs at prediction time. You should ensure that the benefits of integer comparisons outweights the overhead costs.

Mechanism for mapping features

When quantize option is enabled, treelite will collect all thresholds occuring in the tree ensemble model. For each feature, one list will be generated that lists the thresholds in ascending order:

/* example of how per-feature threshold list may look like */

Feature 0:  [1.5, 6.5, 12.5]
Feature 3:  [0.15, 0.35, 1.5]
Feature 6:  [7, 9, 10, 135]

Using these lists, we may convert any data point into integer indices via simple look-ups. For feature 0 in the example above, values will be mapped to integer indices as follows:

Let x be the value of feature 0.

Assign -1 if          x  <  1.5
Assign  0 if          x ==  1.5
Assign  1 if   1.5  < x  <  6.5
Assign  2 if          x ==  6.5
Assign  3 if   6.5  < x  < 12.5
Assign  4 if          x == 12.5
Assign  5 if          x  > 12.5

Let’s look at a specific example of how a floating-point vector gets translated into a vector of integer indices:

feature id   0     1        2      3      4        5      6
            [7, missing, missing, 0.2, missing, missing, 20 ]
         => [3, missing, missing,   1, missing, missing,  5 ]

Since the prediction subroutine still needs to accept floating-point features, the features will be internally converted before actual prediction. If the prediction subroutine looked like below without quantize option,

float predict_margin(const Entry* data) {
  /* ... Run through the trees to compute the leaf output score ... */

  return score;
}

it will now have an extra step of mapping the incoming data vector into integers:

float predict_margin(const Entry* data) {
  /* ... Quantize feature values in data into integer indices   ... */

  /* ... Run through the trees to compute the leaf output score ... */
  return score;
}

Deploying models

After all the hard work you did to train your tree ensemble model, you now have to deploy the model. Deployment refers to distributing your model to other machines and devices so as to make predictions on them. To facilitate the coming discussions, let us define a few terms.

• Host machine : the machine running treelite.
• Target machine : the machine on which predictions will be made. The host machine may or may not be identical to the target machine. In cases where it’s infeasible to install treelite on the target machine, the host and target machines will be necessarily distinct.
• Shared library : a blob of executable subroutines that can be imported by other native applications. Shared libraries will often have file extensions .dll, .so, or .dylib. Going back to the particular context of tree deployment, treelite will produce a shared library containing the prediction subroutine (compiled to native machine code).
• Runtime package : a tiny fraction of the full treelite package, consisting of a few helper functions that lets you easily load shared libraries and make predictions. The runtime is good to have, but on systems lacking Python we can do without it.

In this document, we will document three options for deployment. We will present the programming interface each deployment option presents, as well as its dependencies and requirements.

Option 1: Install treelite on the target machine

If feasible, this option is probably the most convenient. On the target machine, install treelite by running pip:

pip3 install treelite --user

Once treelite is installed, it suffices to follow instructions in First tutorial.

Dependencies and Requirements

The target machine shall meet the following conditions:

• Treelite is installed (either by pip install or by manual compilation, see below)
• One of the following C compiler is available: gcc, clang, Microsoft Visual C++.
• The C compiler supports the following features of the C99 standard: inline functions; declaration of loop variables inside for loop; the expf function in <math.h>; the <stdint.h> header. Recent versions of gcc and clang should work, as well as Microsoft Visual Studio 2013 or newer.
• Python is installed, with version 2.7 or >= 3.4.
• The following Python packages are available: numpy, scipy.sparse.

In addition, if you are using operating systems other than Windows, Mac OS X, and Linux, you would need to compile treelite from the source. To do this, you’ll need git, CMake (>= 3.1), and a C++ compiler that complies with the C++11 standard.

Option 2: Deploy prediction code with the runtime package

With this option, you no longer have to install the full treelite package on the target machine. Only the runtime package and the prediction library will need to be copied to the target machine.

Dependencies and Requirements

The target machine shall meet the following conditions:

• One of the following C++ compiler is available: gcc, clang, Microsoft Visual C++.
• The C++ compiler complies with the C++11 standard. Recent versions of gcc and clang qualifies, as well as Microsoft Visual C++ 2015 or newer.
• When compiling pure C program, the C++ compiler supports the following features of the C99 standard: inline functions; declaration of loop variables inside for loop; the expf function in <math.h>; the <stdint.h> header.
• Python is installed, with version 2.7 or >= 3.4.
• The following Python packages are available: numpy, scipy.sparse.
• CMake 3.1 or newer is installed.
• An archive utility exists that can open a .zip archive.

Deployment instructions

1. On the host machine, install treelite and import your tree ensemble model. You should end up with the model object of type Model.

### Run this block on the **host** machine

import treelite
model = treelite.Model.load('your_model.model', 'xgboost')
# You may also use `from_xgboost` method or the builder class

2. Export your model as a source package by calling the method export_srcpkg() of the Model object. The source package will contain C code representation of the prediction subroutine.

### Continued from the previous code block

# Operating system of the target machine
platform = 'unix'
# C compiler to use to compile prediction code on the target machine
toolchain = 'gcc'
# Save the source package as a zip archive named mymodel.zip
# Later, we'll use this package to produce the library mymodel.so.
model.export_srcpkg(platform=platform, toolchain=toolchain,
                    pkgpath='./mymodel.zip', libname='mymodel.so',
                    verbose=True)

After calling export_srcpkg(), you should be able to find the zip archive named mymodel.zip inside the current working directory.

john.doe@host-machine:/home/john.doe/$ ls .
mymodel.zip   your_model.model

The content of mymodel.zip consists of the header and source files, as well as the Makefile:

john.doe@host-machine:/home/john.doe/$ unzip -l mymodel.zip
Archive:  mymodel.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  11-01-2017 23:11   mymodel/
      167  11-01-2017 23:11   mymodel/Makefile
  4831036  11-01-2017 23:11   mymodel/mymodel.c
      311  11-01-2017 23:11   mymodel/mymodel.h
      109  11-01-2017 23:11   mymodel/recipe.json
---------                     -------
  4831623                     5 files

3. Export the runtime package using the method save_runtime_package():

### Continued from the previous code block
treelite.save_runtime_package(destdir='.')

This command produces the zip archived named treelite_runtime.zip. This archive contains all the necessary files for prediction task.

john.doe@host-machine:/home/john.doe/$ ls .
mymodel.zip   your_model.model   treelite_runtime.zip
john.doe@host-machine:/home/john.doe/$ unzip -l treelite_runtime.zip
Archive:  treelite_runtime.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  10-30-2017 12:23   runtime/
.................full output not shown here.................
      709  09-26-2017 17:58   runtime/src/common/math.h
---------                     -------
    71353                     39 files

4. Now you are ready to deploy the model to the target machine. Copy to the target machine the two archives mymodel.zip (source package) and treelite_runtime.zip (runtime package):

john.doe@host-machine:/home/john.doe/$ sftp john.doe@target-machine
Connected to target-machine.
sftp> put mymodel.zip
Uploading mymodel.zip to /home/john.doe/mymodel.zip
mymodel.zip                             100%  410KB 618.2KB/s   00:00
sftp> put treelite_runtime.zip
Uploading treelite_runtime.zip to /home/john.doe/treelite_runtime.zip
treelite_runtime.zip                    100%   28KB 618.0KB/s   00:00
sftp> quit

5. It is time to move to the target machine. On the target machine, extract both archives mymodel.zip and treelite_runtime.zip:

john.doe@host-machine:/home/john.doe/$ ssh john.doe@target-machine
Last login: Tue Oct 31 00:43:36 2017 from host-machine

john.doe@target-machine:/home/john.doe/$ unzip mymodel.zip
Archive:  mymodel.zip
   creating: mymodel/
  inflating: mymodel/Makefile
  inflating: mymodel/mymodel.c
  inflating: mymodel/mymodel.h
  inflating: mymodel/recipe.json
john.doe@target-machine:/home/john.doe/$ unzip treelite_runtime.zip
Archive:  treelite_runtime.zip
   creating: runtime/
.................full output not shown here.................
  inflating: runtime/src/common/math.h

6. Now build the runtime package: cmake followed by make.

john.doe@target-machine:/home/john.doe/$ cd runtime/build
john.doe@target-machine:/home/john.doe/runtime/build/$ cmake ..
-- The C compiler identification is GNU 7.2.0
-- The CXX compiler identification is GNU 7.2.0
.................full output not shown here.................
-- Configuring done
-- Generating done
-- Build files have been written to: /home/john.doe/runtime/build
john.doe@target-machine:/home/john.doe/runtime/build/$ make
Scanning dependencies of target objtreelite_runtime
[ 50%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/c_api/c_api_common.cc.o
[ 50%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/c_api/c_api_error.cc.o
[ 50%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/c_api/c_api_runtime.cc.o
[ 66%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/logging.cc.o
[ 83%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/predictor.cc.o
[ 83%] Built target objtreelite_runtime
Scanning dependencies of target treelite_runtime
[100%] Linking CXX shared library ../lib/libtreelite_runtime.so
[100%] Built target treelite_runtime
john.doe@target-machine:/home/john.doe/runtime/build/$ cd ../..
john.doe@target-machine:/home/john.doe/$

Note

Building the runtime package on Windows

The example shown assumes the target was UNIX. On Windows, CMake will create a Visual Studio solution file named treelite_runtime.sln. Open it and compile the solution by selecting Build > Build Solution from the menu.

Note

Building the runtime package on Mac OS X

The default clang installation on Mac OS X does not support OpenMP, the language construct for multithreading. To enable multithreading in treelite, we recommend that you install gcc 7.x using Homebrew:

john.doe@target-machine:/home/john.doe/runtime/build/$ brew install gcc@7

After g++ is installed, run CMake again with gcc as the C++ compiler:

john.doe@target-machine:/home/john.doe/runtime/build/$ cmake .. \
                         -DCMAKE_CXX_COMPILER=g++-7 -DCMAKE_C_COMPILER=gcc-7

7. Build the source package. Now only make will do. (On Windows, run NMake instead.)

john.doe@target-machine:/home/john.doe/$ cd mymodel
john.doe@target-machine:/home/john.doe/mymodel/$ make
gcc -c -O3 -o mymodel.o mymodel.c -fPIC -std=c99 -flto -fopenmp
gcc -shared -O3 -o mymodel.so mymodel.o -std=c99 -flto -fopenmp
john.doe@target-machine:/home/john.doe/mymodel/$ ls
Makefile       mymodel.c      mymodel.so
mymodel.h      mymodel.o      recipe.json

Note

Parallel compilation with GNU Make

If you used parallel_comp option to split the model into multiple source files, you can take advantage of parallel compilation. Simply replace make with make -jN, where N is replaced with the number of workers to launch. Setting N too high may result into memory shortage.

8. Temporarily set the environment variable PYTHONPATH to the directory runtime/python, so that the runtime package can be found by the Python interpreter. Then launch an interactive Python session and import the module treelite.runtime. If no error occurs, we are done.

john.doe@target-machine:/home/john.doe/$ set PYTHONPATH=./runtime/python/
john.doe@target-machine:/home/john.doe/$ ipython
Python 3.6.2 (default, Jul 17 2017, 16:44:45)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import treelite.runtime

Prediction instructions

Finally, we are ready to make predictions, per instructions given in First tutorial. Don’t forget to set PYTHONPATH before running the following script:

import treelite.runtime
import numpy as np
# Load data
X = np.load('data.npy')
batch = Batch.from_npy2d(X)
# Load predictor
predictor = treelite.runtime.Predictor('./mymodel/mymodel.so', verbose=True)
out_pred = predictor.predict(batch)

Option 3: Deploy prediciton code only

With this option, neither Python nor a C++ compiler is required. You should be able to adopt this option using any basic installation of UNIX-like operating systems.

Dependencies and Requirements

The target machine shall meet the following conditions:

• A C compiler is available.
• The C compiler supports the following features of the C99 standard: inline functions; declaration of loop variables inside for loop; the expf function in <math.h>; the <stdint.h> header.
• GNU Make or Microsoft NMake is installed.
• An archive utility exists that can open a .zip archive.

Deployment instructions

1. On the host machine, install treelite and import your tree ensemble model. You should end up with the model object of type Model.

### Run this block on the **host** machine

import treelite
model = treelite.Model.load('your_model.model', 'xgboost')
# You may also use `from_xgboost` method or the builder class

2. Export your model as a source package by calling the method export_srcpkg() of the Model object. The source package will contain C code representation of the prediction subroutine.

### Continued from the previous code block

# Operating system of the target machine
platform = 'unix'
# C compiler to use to compile prediction code on the target machine
toolchain = 'gcc'
# Save the source package as a zip archive named mymodel.zip
# Later, we'll use this package to produce the library mymodel.so.
model.export_srcpkg(platform=platform, toolchain=toolchain,
                    pkgpath='./mymodel.zip', libname='mymodel.so',
                    verbose=True)

Note

On the value of toolchain

Treelite supports only three toolchain configurations (‘msvc’, ‘gcc’, ‘clang’) for which it generates Makefiles. If you are using a compiler other than these three, you will have to write your own Makefile. For now, just set toolchain='gcc' and move on.

After calling export_srcpkg(), you should be able to find the zip archive named mymodel.zip inside the current working directory.

john.doe@host-machine:/home/john.doe/$ ls .
mymodel.zip   your_model.model

The content of mymodel.zip consists of the header and source files, as well as the Makefile:

john.doe@host-machine:/home/john.doe/$ unzip -l mymodel.zip
Archive:  mymodel.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  11-01-2017 23:11   mymodel/
      167  11-01-2017 23:11   mymodel/Makefile
  4831036  11-01-2017 23:11   mymodel/mymodel.c
      311  11-01-2017 23:11   mymodel/mymodel.h
      109  11-01-2017 23:11   mymodel/recipe.json
---------                     -------
  4831623                     5 files

3. Now you are ready to deploy the model to the target machine. Copy to the target machine the archive mymodel.zip (source package).

john.doe@host-machine:/home/john.doe/$ sftp john.doe@target-machine
Connected to target-machine.
sftp> put mymodel.zip
Uploading mymodel.zip to /home/john.doe/mymodel.zip
mymodel.zip                             100%  410KB 618.2KB/s   00:00
sftp> quit

4. It is time to move to the target machine. On the target machine, extract the archive mymodel.zip:

john.doe@host-machine:/home/john.doe/$ ssh john.doe@target-machine
Last login: Tue Oct 31 00:43:36 2017 from host-machine

john.doe@target-machine:/home/john.doe/$ unzip mymodel.zip
Archive:  mymodel.zip
   creating: mymodel/
  inflating: mymodel/Makefile
  inflating: mymodel/mymodel.c
  inflating: mymodel/mymodel.h
  inflating: mymodel/recipe.json

5. Build the source package (using GNU Make or NMake).

john.doe@target-machine:/home/john.doe/$ cd mymodel
john.doe@target-machine:/home/john.doe/mymodel/$ make
gcc -c -O3 -o mymodel.o mymodel.c -fPIC -std=c99 -flto -fopenmp
gcc -shared -O3 -o mymodel.so mymodel.o -std=c99 -flto -fopenmp
john.doe@target-machine:/home/john.doe/mymodel/$ ls
Makefile       mymodel.c      mymodel.so
mymodel.h      mymodel.o      recipe.json

Note

Parallel compilation with GNU Make

If you used parallel_comp option to split the model into multiple source files, you can take advantage of parallel compilation. Simply replace make with make -jN, where N is replaced with the number of workers to launch. Setting N too high may result into memory shortage.

Note

Using other compilers

If you are using a compiler other than gcc, clang, or Microsoft Visual C++, you will need to compose your own Makefile. Open the Makefile and make necessary changes.

Prediction instructions

The prediction library provides the function predict with the following signature:

float predict(union Entry* data, int pred_margin);

Here, the argument data must be an array of length M, where M is the number of features used in the tree ensemble. The data array stores all the feature values of a single row. To indicate presence or absence of a feature value, we use the union type Entry, which defined as

union Entry {
  int missing;
  float fvalue;
};

For missing values, we set the missing field to -1. For non-missing ones, we set the fvalue field to the feature value. The total number of features is given by the function

size_t get_num_feature(void);

Let’s look at an example. We’d start by initializing the array inst, a dense aray to hold feature values of a single data row:

/* number of features */
const size_t num_feature = get_num_feature();
/* inst: dense vector storing feature values */
union Entry* inst = malloc(sizeof(union Entry) * num_feature);
/* clear inst with all missing values */
for (i = 0; i < num_feature; ++i) {
  inst[i].missing = -1;
}

Before calling the function predict, the array inst needs to be initialized with missing and present feature values. The following peudocode illustrates the idea:

For each data row rid:
  inst[i].missing == -1 for every i, assuming all features lack values

  For each feature i for which the data row in fact has a feature value:
    Set inst[i].fvalue = [feature value], to indicate presence

  Call predict(inst, 0) and get prediction for the data row rid

  For each feature i for which the row has a feature value:
    Set inst[i].missing = -1, to prepare for next row (rid + 1)

The task is not too difficult as long as the input data is given as a particular form of sparse matrix: the Compressed Sparse Row format. The sparse matrix consists of three arrays:

• val stores nonzero entries in row-major order.
• col_ind stores column indices of the entries in val. The expression col_ind[i] indicates the column index of the i th entry val[i].
• row_ptr stores the locations in val that start and end data rows. The i th data row is given by the array slice val[row_ptr[i]:row_ptr[i+1]].

/* nrow : number of data rows */
for (rid = 0; rid < nrow; ++rid) {
  ibegin = row_ptr[rid];
  iend = row_ptr[rid + 1];
  /* Fill nonzeros */
  for (i = ibegin; i < iend; ++i) {
    inst[col_ind[i]].fvalue = val[i];
  }
  out_pred[rid] = predict(inst, 0);
  /* Drop nonzeros */
  for (i = ibegin; i < iend; ++i) {
    inst[col_ind[i]].missing = -1;
  }
}

It only remains to create three arrays val, col_ind, and row_ptr. You may want to use a third-pary library here to read from a SVMLight format. For now, we’ll punt the issue of loading the input data and write it out as constants in the program:

#include <stdio.h>
#include <stdlib.h>
#include "mymodel.h"

int main(void) {
  /* 5x13 "sparse" matrix, in CSR format
     [[ 0.  ,  0.  ,  0.68,  0.99,  0.  ,  0.11,  0.  ,  0.82,  0.  ,
        0.  ,  0.  ,  0.  ,  0.  ],
      [ 0.  ,  0.  ,  0.99,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,
        0.61,  0.  ,  0.  ,  0.  ],
      [ 0.02,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,
        0.  ,  0.  ,  0.  ,  0.  ],
      [ 0.  ,  0.  ,  0.36,  0.  ,  0.82,  0.  ,  0.  ,  0.57,  0.  ,
        0.  ,  0.  ,  0.  ,  0.75],
      [ 0.47,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,
        0.  ,  0.  ,  0.45,  0.  ]]
  */
  const float val[] = {0.68, 0.99, 0.11, 0.82, 0.99, 0.61, 0.02, 0.36, 0.82,
                       0.57, 0.75, 0.47, 0.45};
  const size_t col_ind[] = {2, 3, 5, 7, 2, 9, 0, 2, 4, 7, 12, 0, 11};
  const size_t row_ptr[] = {0, 4, 6, 7, 11, 13};
  const size_t nrow = 5;
  const size_t ncol = 13;

  /* number of features */
  const size_t num_feature = get_num_feature();
  /* inst: dense vector storing feature values */
  union Entry* inst = malloc(sizeof(union Entry) * num_feature);
  float* out_pred = malloc(sizeof(float) * nrow);
  size_t rid, ibegin, iend, i;

  /* clear inst with all missing */
  for (i = 0; i < num_feature; ++i) {
    inst[i].missing = -1;
  }

  for (rid = 0; rid < nrow; ++rid) {
    ibegin = row_ptr[rid];
    iend = row_ptr[rid + 1];
    /* Fill nonzeros */
    for (i = ibegin; i < iend; ++i) {
      inst[col_ind[i]].fvalue = val[i];
    }
    out_pred[rid] = predict(inst, 0);
    /* Drop nonzeros */
    for (i = ibegin; i < iend; ++i) {
      inst[col_ind[i]].missing = -1;
    }
    printf("pred[%zu] = %f\n", rid, out_pred[rid]);
  }
  free(inst);
  free(out_pred);
  return 0;
}

Save the program as a .c file and put it in the same directory mymodel/. To link the program against the prediction library mymodel.so, simply run

gcc -o myprog myprog.c mymodel.so -I. -std=c99

As long as the program myprog is in the same directory of the prediction library mymodel.so, we’ll be good to go.

A sample output:

pred[0] = 44.880001
pred[1] = 44.880001
pred[2] = 44.880001
pred[3] = 42.670002
pred[4] = 44.880001

Specifying models using model builder

Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.

Using XGBoost or LightGBM for training? Read this document instead.

What is the model builder?

The ModelBuilder class is a tool used to specify decision tree ensembles programmatically. Each tree ensemble is represented as follows:

• Each Tree object is a dictionary of nodes indexed by unique integer keys.
• A node is either a leaf node or a test node. A test node specifies its left and right children by their integer keys in the tree dictionary.
• Each ModelBuilder object is a list of Tree objects.

A toy example

Consider the following tree ensemble, consisting of two regression trees:

Note

Provision for missing data: default directions

Decision trees in treelite accomodate missing data by indicating the default direction for every test node. In the diagram above, the default direction is indicated by label “Missing.” For instance, the root node of the first tree shown above will send to the left all data points that lack values for feature 0.

For now, let’s assume that we’ve somehow found optimal choices of default directions at training time. For detailed instructions for actually deciding default directions, see Section 3.4 of the XGBoost paper.

Let us construct this ensemble using the model builder. First step is to assign unique integer key to each node. In the following diagram, integer keys are indicated in red. Note that integer keys need to be unique only within the same tree.

Next, we create a model builder object by calling the constructor for ModelBuilder, with an num_feature argument indicating the total number of features used in the ensemble:

import treelite
builder = treelite.ModelBuilder(num_feature=3)

We also create a tree object; it will represent the first tree in the ensemble.

# to represent the first tree
tree = treelite.ModelBuilder.Tree()

The first tree has five nodes, each of which is to be inserted into the tree one at a time. The syntax for node insertion is as follows:

tree[0]   # insert a new node with key 0

Once a node has been inserted, we can refer to it by writing

tree[0]   # refer to existing node #0

The meaning of the expression tree[0] thus depends on whether the node #0 exists in the tree or not.

We may combine node insertion with a function call to specify its content. For instance, node #0 is a test node, so we call set_numerical_test_node():

# Node #0: feature 0 < 5.0 ? (default direction left)
tree[0].set_numerical_test_node(feature_id=0,
                                opname='<',
                                threshold=5.0,
                                default_left=True,
                                left_child_key=1,
                                right_child_key=2)

On the other hand, node #2 is a leaf node, so call set_leaf_node() instead:

# Node #2: leaf with output +0.6
tree[2].set_leaf_node(0.6)

Let’s go ahead and specify the other three nodes:

# Node #1: feature 2 < -3.0 ? (default direction right)
tree[1].set_numerical_test_node(feature_id=2,
                                opname='<',
                                threshold=-3.0,
                                default_left=False,
                                left_child_key=3,
                                right_child_key=4)
# Node #3: leaf with output -0.4
tree[3].set_leaf_node(-0.4)
# Node #4: leaf with output +1.2
tree[4].set_leaf_node(1.2)

We must indicate which node is the root:

# Set node #0 as root
tree[0].set_root()

We are now done with the first tree. We insert it with the model builder by calling append(). (Recall that the model builder is really a list of tree objects, hence the method name append.)

# Insert the first tree into the ensemble
builder.append(tree)

The second tree is constructed analogously:

tree2 = treelite.ModelBuilder.Tree()
# Node #0: feature 1 < 2.5 ? (default direction right)
tree2[0].set_numerical_test_node(feature_id=1,
                                 opname='<',
                                 threshold=2.5,
                                 default_left=False,
                                 left_child_key=1,
                                 right_child_key=2)
# Set node #0 as root
tree2[0].set_root()
# Node #1: leaf with output +1.6
tree2[1].set_leaf_node(1.6)
# Node #2: feature 2 < -1.2 ? (default direction left)
tree2[2].set_numerical_test_node(feature_id=2,
                                 opname='<',
                                 threshold=-1.2,
                                 default_left=True,
                                 left_child_key=3,
                                 right_child_key=4)
# Node #3: leaf with output +0.1
tree2[3].set_leaf_node(0.1)
# Node #4: leaf with output -0.3
tree2[4].set_leaf_node(-0.3)

# Insert the second tree into the ensemble
builder.append(tree2)

We are now done building the member trees. The last step is to call commit() to finalize the ensemble into a Model object:

# Finalize and obtain Model object
model = builder.commit()

Note

Difference between ModelBuilder and Model objects

Why does treelite require one last step of “committing”? All Model objects are immutable; once constructed, they cannot be modified at all. So you won’t be able to add a tree or a node to an existing Model object, for instance. On the other hand, ModelBuilder objects are mutable, so that you can iteratively build trees.

To ensure we got all details right, we can examine the resulting C program.

model.compile(dirpath='./test')
with open('./test/test.c', 'r') as f:
  for line in f.readlines():
    print(line, end='')

which produces the output

/* Other functions omitted for space consideration */
float predict_margin(union Entry* data) {
  float sum = 0.0f;
  if (!(data[0].missing != -1) || data[0].fvalue < 5) {
    if ( (data[2].missing != -1) && data[2].fvalue < -3) {
      sum += (float)-0.4;
    } else {
      sum += (float)1.2;
    }
  } else {
    sum += (float)0.6;
  }
  if ( (data[1].missing != -1) && data[1].fvalue < 2.5) {
    sum += (float)1.6;
  } else {
    if (!(data[2].missing != -1) || data[2].fvalue < -1.2) {
      sum += (float)0.1;
    } else {
      sum += (float)-0.3;
    }
  }
  return sum + (0);
}

The toy example has been helpful as an illustration, but it is impractical to manually specify nodes for real-world ensemble models. The following section will show us how to automate the tree building process. We will look at scikit-learn in particular.

Using the model builder to interface with scikit-learn

Scikit-learn (scikit-learn/scikit-learn) is a Python machine learning package known for its versatility and ease of use. It supports a wide variety of models and algorithms.

Treelite will be able to work with any decision tree ensemble models produced by scikit-learn. In particular, it will be able to work with

• sklearn.ensemble.RandomForestRegressor
• sklearn.ensemble.RandomForestClassifier
• sklearn.ensemble.GradientBoostingRegressor
• sklearn.ensemble.GradientBoostingClassifier

Note

Why scikit-learn? How about other packages?

We had to pick a specific example for programmatic tree construction, so we chose scikit-learn. If you’re using another package, don’t lose heart. As you read through the rest of section, notice how specific pieces of information about the tree ensemble model are being extracted. As long as your choice of package exposes equivalent information, you’ll be able to adapt the example to your needs.

Note

In a hurry? Try the gallery module

The rest of this document explains in detail how to import scikit-learn models using the builder class. If you prefer to skip all the gory details, simply import the module treelite.gallery.sklearn.

import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)

Note

Adaboost ensembles not yet supported

Treelite currently does not support weighting of member trees, so you won’t be able to use Adaboost ensembles.

Regression with RandomForestRegressor

Let’s start with the Boston house prices dataset, a regression problem. (Classification problems are somewhat trickier, so we’ll save them for later.)

We’ll be using RandomForestRegressor, a random forest for regression. A random forest is an ensemble of decision trees that are independently trained on random samples from the training data. See this page for more details. For now, just remember to specify random_forest=True in the ModelBuilder constructor.

import sklearn.datasets
import sklearn.ensemble
# Load the Boston housing dataset
X, y = sklearn.datasets.load_boston(return_X_y=True)
# Train a random forest regressor with 10 trees
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)

We shall programmatically construct Tree objects from internal attributes of the scikit-learn model. We only need to define a few helper functions.

For the rest of sections, we’ll be diving into lots of details that are specific to scikit-learn. Many details have been adopted from this reference page.

The function process_model() takes in a scikit-learn ensemble object and returns the completed Model object:

def process_model(sklearn_model):
  # Initialize treelite model builder
  # Set random_forest=True for random forests
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  random_forest=True)

  # Iterate over individual trees
  for i in range(sklearn_model.n_estimators):
    # Process the i-th tree and add to the builder
    # process_tree() to be defined later
    builder.append( process_tree(sklearn_model.estimators_[i].tree_,
                                 sklearn_model) )

  return builder.commit()

The usage of this function is as follows:

model = process_model(clf)

We won’t have space here to discuss all internals of scikit-learn objects, but a few details should be noted:

• The attribute n_features_ stores the number of features used anywhere in the tree ensemble.
• The attribute n_estimators stores the number of member trees.
• The attribute estimators_ is an array of handles that store the individual member trees. To access the object for the i-th tree, write estimators_[i].tree_. This object will be passed to the function process_tree().

The function process_tree() takes in a single scikit-learn tree object and returns an object of type Tree:

def process_tree(sklearn_tree, sklearn_model):
  treelite_tree = treelite.ModelBuilder.Tree()
  # Node #0 is always root for scikit-learn decision trees
  treelite_tree[0].set_root()

  # Iterate over each node: node ID ranges from 0 to [node_count]-1
  for node_id in range(sklearn_tree.node_count):
    process_node(treelite_tree, sklearn_tree, node_id, sklearn_model)

  return treelite_tree

Explanations:

• The attribute node_count stores the number of nodes in the decision tree.
• Each node in the tree has a unique ID ranging from 0 to [node_count]-1.

The function process_node() determines whether each node is a leaf node or a test node. It does so by looking at the attribute children_left: If the left child of the node is set to -1, that node is thought to be a leaf node.

def process_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  if sklearn_tree.children_left[node_id] == -1:  # leaf node
    process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model)
  else:                                          # test node
    process_test_node(treelite_tree, sklearn_tree, node_id, sklearn_model)

The function process_test_node() extracts the content of a test node and passes it to the Tree object that is being constructed.

def process_test_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  # Initialize the test node with given node ID
  treelite_tree[node_id].set_numerical_test_node(
                        feature_id=sklearn_tree.feature[node_id],
                        opname='<=',
                        threshold=sklearn_tree.threshold[node_id],
                        default_left=True,
                        left_child_key=sklearn_tree.children_left[node_id],
                        right_child_key=sklearn_tree.children_right[node_id])

Explanations:

• The attribute feature is the array containing feature indices used in test nodes.
• The attribute threshold is the array containing threshold values used in test nodes.
• All tests are in the form of [feature value] <= [threshold].
• The attributes children_left and children_right together store children’s IDs for test nodes.

Note

Scikit-learn and missing data

Scikit-learn handles missing data differently than XGBoost and treelite. Before training an ensemble model, you’ll have to impute missing values. For this reason, test nodes in scikit-learn tree models will contain no “default direction.” We will assign default_left=True arbitrarily for test nodes to keep treelite happy.

The function process_leaf_node() defines a leaf node:

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  # The `value` attribute stores the output for every leaf node.
  leaf_value = sklearn_tree.value[node_id].squeeze()
  # Initialize the leaf node with given node ID
  treelite_tree[node_id].set_leaf_node(leaf_value)

Let’s test it out:

model = process_model(clf)
model.export_lib(libpath='./libtest.dylib', toolchain='gcc', verbose=True)

import treelite.runtime
predictor = treelite.runtime.Predictor(libpath='./libtest.dylib')
predictor.predict(treelite.runtime.Batch.from_npy2d(X))

Regression with GradientBoostingRegressor

Gradient boosting is an algorithm where decision trees are trained one at a time, ensuring that latter trees complement former trees. See this page for more details. Treelite makes distinction between random forests and gradient boosted trees by the value of random_forest flag in the ModelBuilder constructor.

Note

Set init='zero' to ensure compatibility

To make sure that the gradient boosted model is compatible with treelite, make sure to set init='zero' in the GradientBoostingRegressor constructor. This ensures that the compiled prediction subroutine will produce the correct prediction output. Gradient boosting models trained without specifying init='zero' in the constructor are NOT supported by treelite!

# Gradient boosting regressor
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingRegressor(n_estimators=10,
                                                 init='zero')
clf.fit(X, y)

We will recycle most of the helper code we wrote earlier. Only two functions will need to be modified:

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  # Check for init='zero'
  if sklearn_model.init != 'zero':
    raise Exception("Gradient boosted trees must be trained with "
                    "the option init='zero'")
  # Initialize treelite model builder
  # Set random_forest=False for gradient boosted trees
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  random_forest=False)
  for i in range(sklearn_model.n_estimators):
    # Process i-th tree and add to the builder
    builder.append( process_tree(sklearn_model.estimators_[i][0].tree_,
                                 sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  leaf_value = sklearn_tree.value[node_id].squeeze()
  # Need to shrink each leaf output by the learning rate
  leaf_value *= sklearn_model.learning_rate
  # Initialize the leaf node with given node ID
  treelite_tree[node_id].set_leaf_node(leaf_value)

Some details specific to GradientBoostingRegressor:

• To indicate the use of gradient boosting (as opposed to random forests), we set random_forest=False in the ModelBuilder constructor.
• Each tree object is now accessed with the expression estimators_[i][0].tree_, as estimators_[i] returns an array consisting of a single tree (i-th tree).
• Each leaf output in gradient boosted trees are “unscaled”: it needs to be scaled by the learning rate.

Let’s test it:

# Convert to treelite model
model = process_model(clf)
# Generate shared library
model.export_lib(libpath='./libtest2.dylib', toolchain='gcc', verbose=True)
# Make prediction with predictor
predictor = treelite.runtime.Predictor(libpath='./libtest2.dylib')
predictor.predict(treelite.runtime.Batch.from_npy2d(X))

Binary Classification with RandomForestClassifier

For binary classification, let’s use the digits dataset. We will take 0’s and 1’s from the dataset and treat 0’s as the negative class and 1’s as the positive.

# load a binary classification problem
# Set n_class=2 to produce two classes
digits = sklearn.datasets.load_digits(n_class=2)
X, y = digits['data'], digits['target']
# Should print [0 1]
print(np.unique(y))

# Train a random forest classifier
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)

Random forest classifiers in scikit-learn store frequency counts for the positive and negative class. For instance, a leaf node may output a set of counts

[ 100, 200 ]

which indicates the following:

• 300 data points in the training set “belong” to this leaf node, in the sense that they all satisfy the precise sequence of conditions leading to that particular leaf node. The picture below shows that each leaf node represents a unique sequence of conditions:

• 100 of them are labeled negative; and
• the remaining 200 are labeled positive.

Again, most of the helper functions may be re-used; only two functions need to be rewritten. Explanation will follow after the code:

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  random_forest=True)
  for i in range(sklearn_model.n_estimators):
    # Process i-th tree and add to the builder
    builder.append( process_tree(sklearn_model.estimators_[i].tree_,
                                 sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  # Get counts for each label (+/-) at this leaf node
  leaf_count = sklearn_tree.value[node_id].squeeze()
  # Compute the fraction of positive data points at this leaf node
  fraction_positive = float(leaf_count[1]) / leaf_count.sum()
  # The fraction above is now the leaf output
  treelite_tree[node_id].set_leaf_node(fraction_positive)

As noted earlier, we access the frequency counts at each leaf node, reading the value attribute of each tree. Then we compute the fraction of positive data points with respect to all training data points belonging to the leaf. This fraction then becomes the leaf output. This way, leaf nodes now produce single numbers rather than frequency count arrays.

Why did we have to compute a fraction? For binary classification, treelite expects each tree to produce a single number output. At prediction time, the outputs from the member trees will get averaged to produce the final prediction, which is also a single number. By setting the positive fraction as the leaf output, we ensure that the final prediction is a proper probability value. For instance, if an ensemble consisting of 5 trees produces the following set of outputs

Tree 0    0.1
Tree 1    0.7
Tree 2    0.4
Tree 3    0.3
Tree 4    0.7

then the final prediction will be 0.44, which we interpret as 44% probability for the positive class.

Multi-class Classification with RandomForestClassifier

Let’s use the digits dataset again, this time with 4 classes (i.e. 0’s, 1’s, 2’s, and 3’s).

# Load a multi-class classification problem
# Set n_class=4 to produce four classes
digits = sklearn.datasets.load_digits(n_class=4)
X, y = digits['data'], digits['target']
# Should print [0 1 2 3]
print(np.unique(y))

# Train a random forest classifier
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)

Random forest classifiers in scikit-learn store frequency counts (see the explanation in the previous section). For instance, a leaf node may output a set of counts

[ 100, 400, 300, 200 ]

which shows that the total of 1000 training data points belong to this leaf node and that 100, 400, 300, and 200 of them are labeled class 0, 1, 2, and 3, respectively.

We will have to re-write the process_leaf_node() function to accomodate multiple classes.

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  # Must specify num_output_group and pred_transform
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  num_output_group=sklearn_model.n_classes_,
                                  random_forest=True,
                                  pred_transform='identity_multiclass')
  for i in range(sklearn_model.n_estimators):
    # Process i-th tree and add to the builder
    builder.append( process_tree(sklearn_model.estimators_[i].tree_,
                                 sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  # Get counts for each label class at this leaf node
  leaf_count = sklearn_tree.value[node_id].squeeze()
  # Compute the probability distribution over label classes
  prob_distribution = leaf_count / leaf_count.sum()
  # The leaf output is the probability distribution
  treelite_tree[node_id].set_leaf_node(prob_distribution)

The process_leaf_node() function is quite similar to what we had for the binary classification case. Only difference is that, instead of computing the fraction of the positive class, we compute the probability distribution for all possible classes. Each leaf node thus will store the probability distribution of possible class outcomes.

The process_model() function is also similar to what we had before. The crucial difference is the existence of parameters num_output_group and pred_transform. The num_output_group parameter is used only for multi-class classification: it should store the number of classes (in this example, 4). The pred_transform parameter should be set to 'identity_multiclass', to indicate that the prediction should be made simply by averaging the probability distribution produced by each leaf node. (Leaf outputs are averaged rather than summed because we set random_forest=True.) For instance, if an ensemble consisting of 3 trees produces the following set of outputs

Tree 0    [ 0.5, 0.5, 0.0, 0.0 ]
Tree 1    [ 0.1, 0.5, 0.3, 0.1 ]
Tree 2    [ 0.2, 0.5, 0.2, 0.1 ]

then the final prediction will be the average [ 0.26666667, 0.5, 0.16666667, 0.06666667 ], which indicates 26.7% probability for the first class, 50.0% for the second, 16.7% for the third, and 6.7% for the fourth.

Binary Classification with GradientBoostingClassifier

We use the digits dataset. We will take 0’s and 1’s from the dataset and treat 0’s as the negative class and 1’s as the positive.

# Load a binary classification problem
# Set n_class=2 to produce two classes
digits = sklearn.datasets.load_digits(n_class=2)
X, y = digits['data'], digits['target']
# Should print [0 1]
print(np.unique(y))

# Train a gradient boosting classifier
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingClassifier(n_estimators=10,
                                                  init='zero')
clf.fit(X, y)

Note

Set init='zero' to ensure compatibility

To make sure that the gradient boosted model is compatible with treelite, make sure to set init='zero' in the GradientBoostingClassifier constructor. This ensures that the compiled prediction subroutine will produce the correct prediction output. Gradient boosting models trained without specifying init='zero' in the constructor are NOT supported by treelite!

Here are the functions process_model() and process_leaf_node() for this scenario:

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  # Check for init='zero'
  if sklearn_model.init != 'zero':
    raise Exception("Gradient boosted trees must be trained with "
                    "the option init='zero'")
  # Initialize treelite model builder
  # Set random_forest=False for gradient boosted trees
  # Set pred_transform='sigmoid' to obtain probability predictions
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  random_forest=False,
                                  pred_transform='sigmoid')
  for i in range(sklearn_model.n_estimators):
    # Process i-th tree and add to the builder
    builder.append( process_tree(sklearn_model.estimators_[i][0].tree_,
                                 sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  leaf_value = sklearn_tree.value[node_id].squeeze()
  # Need to shrink each leaf output by the learning rate
  leaf_value *= sklearn_model.learning_rate
  # Initialize the leaf node with given node ID
  treelite_tree[node_id].set_leaf_node(leaf_value)

Some details specific to GradientBoostingClassifier:

• To indicate the use of gradient boosting (as opposed to random forests), we set random_forest=False in the ModelBuilder constructor.
• Each tree object is now accessed with the expression estimators_[i][0].tree_, as estimators_[i] returns an array consisting of a single tree (i-th tree).
• Each leaf output in gradient boosted trees are “unscaled”: it needs to be scaled by the learning rate.

In addition, we specify the parameter pred_transform='sigmoid' so that the final prediction yields the probability for the positive class. For example, suppose that an ensemble consisting of 4 trees produces the following set of outputs:

Tree 0    +0.5
Tree 1    -2.3
Tree 2    +1.5
Tree 3    -1.5

Unlike the random forest example earlier, we do not assume that each leaf output is between 0 and 1; it can be any real number, negative or positive. These numbers are referred to as margin scores, to distinguish them from probabilities.

To obtain the probability for the positive class, we first sum the margin scores (outputs) from the member trees.

Tree 0    +0.5
Tree 1    -2.3
Tree 2    +1.5
Tree 3    -1.5
--------------
Total     -1.8

Then we apply the sigmoid function:

The resulting value is the final prediction. You may interpret this value as a probability. For the particular example, the sigmoid value of -1.8 is 0.14185106, which we interpret as 14.2% probability for the positive class.

Multi-class Classification with GradientBoostingClassifier

Let’s use the digits dataset again, this time with 4 classes (i.e. 0’s, 1’s, 2’s, and 3’s).

# Load a multi-class classification problem
# Set n_class=4 to produce four classes
digits = sklearn.datasets.load_digits(n_class=4)
X, y = digits['data'], digits['target']
# Should print [0 1 2 3]
print(np.unique(y))

# Train a gradient boosting classifier
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingClassifier(n_estimators=10,
                                                  init='zero')
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)

Note

Set init='zero' to ensure compatibility

To make sure that the gradient boosted model is compatible with treelite, make sure to set init='zero' in the GradientBoostingClassifier constructor. This ensures that the compiled prediction subroutine will produce the correct prediction output. Gradient boosting models trained without specifying init='zero' in the constructor are NOT supported by treelite!

Here are the functions process_model() and process_leaf_node() for this scenario:

# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions

def process_model(sklearn_model):
  # Check for init='zero'
  if sklearn_model.init != 'zero':
    raise Exception("Gradient boosted trees must be trained with "
                    "the option init='zero'")
  # Initialize treelite model builder
  # Set random_forest=False for gradient boosted trees
  # Set num_output_group for multiclass classification
  # Set pred_transform='softmax' to obtain probability predictions
  builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
                                  num_output_group=sklearn_model.n_classes_,
                                  random_forest=False,
                                  pred_transform='softmax')
  # Process [number of iterations] * [number of classes] trees
  for i in range(sklearn_model.n_estimators):
    for k in range(sklearn_model.n_classes_):
      builder.append( process_tree(sklearn_model.estimators_[i][k].tree_,
                                   sklearn_model) )

  return builder.commit()

def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
  leaf_value = sklearn_tree.value[node_id].squeeze()
  # Need to shrink each leaf output by the learning rate
  leaf_value *= sklearn_model.learning_rate
  # Initialize the leaf node with given node ID
  treelite_tree[node_id].set_leaf_node(leaf_value)

The process_leaf_node() function is identical to one in the previous section: as before, each leaf node produces a single real-number output.

On the other hand, the process_model() function needs some explanation. First of all, the attribute estimators_ of the scikit-learn model object now stores output groups, which are simply groups of decision trees. The expression estimators_[i] thus refers to the i th output group. Each output group contains as many trees as there are label classes. For the digits example with 4 label classes, we’d have 4 trees for each output group: estimators_[i][0], estimators_[i][1], estimators_[i][2], and estimators_[i][3]. Since there are as many output groups as the number of iterations used for training, the total number of member trees is [number of iterations] * [number of classes]. We have to call append() once for each member tree; hence the use of nested loop.

We also set pred_transform='softmax', which indicates the way margin outputs should be transformed to produce probability predictions. Let us look at a concrete example: suppose we train an ensemble model with 3 rounds of gradient boosting. It would produce a total of 12 decision trees (3 rounds * 4 classes). Suppose also that, given a single test data point, the model produces the following set of margins:

Output group 0:
  Tree  0 produces  +0.5
  Tree  1 produces  +1.5
  Tree  2 produces  -2.3
  Tree  3 produces  -1.5
Output group 1:
  Tree  4 produces  +0.1
  Tree  5 produces  +0.7
  Tree  6 produces  +1.5
  Tree  7 produces  -0.9
Output group 2:
  Tree  8 produces  -0.1
  Tree  9 produces  +0.3
  Tree 10 produces  -0.7
  Tree 11 produces  +0.2

How do we compute probabilities for each of the 4 classes? First, we compute the sum of the margin scores for each output group:

Output group 0:
  Tree  0 produces  +0.5
  Tree  1 produces  +1.5
  Tree  2 produces  -2.3
  Tree  3 produces  -1.5
  ----------------------
  SUBTOTAL          -1.8
Output group 1:
  Tree  4 produces  +0.1
  Tree  5 produces  +0.7
  Tree  6 produces  +1.5
  Tree  7 produces  -0.9
  ----------------------
  SUBTOTAL          +1.4
Output group 2:
  Tree  8 produces  -0.1
  Tree  9 produces  +0.3
  Tree 10 produces  -0.7
  Tree 11 produces  +0.2
  ----------------------
  SUBTOTAL          -0.3

The vector [-1.8, +1.4, -0.3] consisting of the subtotals quantifies the relative likelihood of the label classes. Since the second element (1.4) is the largest, the second class must be the most likely outcome for the particular data point. This vector is not yet a probability distribution, since its elements do not sum to 1.

The softmax function transforms any real-valued vector into a probability distribution as follows:

1. Apply the exponential function (exp) to every element in the vector. This step ensures that every element is positive.
2. Divide every element by the sum over the vector. This step is also known as normalizing the vector. After thie step, the elements of the vector will add up to 1.

Let’s walk through the steps with the vector [-1.8, +1.4, -0.3]. Applying the exponential function is simple with Python:

x = np.exp([-1.8, +1.4, -0.3])
print(x)

which yields

[ 0.16529889  4.05519997  0.74081822]

Note that every element is now positive. Then we normalize the vector by writing

x = x / x.sum()
print(x)

which gives a proper probability distribution:

[ 0.03331754  0.8173636   0.14931886]

We can now interpret the result as giving 3.3% probability for the first class, 81.7% probability for the second, and 14.9% probability for the third.

Specifying models using Protocol Buffers

Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.

Using XGBoost or LightGBM for training? Read this document instead.

What is Protocol Buffers?

Protocol Buffers (google/protobuf) is a widely used mechanism to serialize structured data. You may specify your ensemble model according to the specification src/tree.proto. Depending on the package you used to train the model, it may take some effort to express the model in terms of the given spec. See this helpful guide on reading and writing serialized messages.

To import models that had been serialized with Protocol Buffers, use the load() method with argument format='protobuf':

# model had been saved to a file named my_model.bin
# notice the second argument format='protobuf'
model = Model.load('my_model.bin', format='protobuf')

Treelite API

API of treelite Python package.

treelite: a framework to optimize decision tree ensembles for fast prediction

class treelite.DMatrix(data, data_format=None, missing=None, feature_names=None, feature_types=None, verbose=False, nthread=None)

Data matrix used in treelite.

Parameters:

• data (str / numpy.ndarray / scipy.sparse.csr_matrix / pandas.DataFrame) – Data source. When data is str type, it indicates that data should be read from a file.
• data_format (str, optional) – Format of input data file. Applicable only when data is read from a file. If missing, the svmlight (.libsvm) format is assumed.
• missing (float, optional) – Value in the data that represents a missing entry. If set to None, numpy.nan will be used.
• verbose (bool, optional) – Whether to print extra messages during construction
• feature_names (list, optional) – Human-readable names for features
• feature_types (list, optional) – Types for features
• nthread (int, optional) – Number of threads

class treelite.Model(handle=None)

Decision tree ensemble model

Parameters:handle (ctypes.c_void_p, optional) – Initial value of model handle

compile(dirpath, params=None, compiler='ast_native', verbose=False)

Generate prediction code from a tree ensemble model. The code will be C99 compliant. One header file (.h) will be generated, along with one or more source files (.c). Use create_shared() method to package prediction code as a dynamic shared library (.so/.dll/.dylib).

Parameters:

• dirpath (str) – directory to store header and source files
• params (dict, optional) – parameters for compiler. See this page for the list of compiler parameters.
• compiler (str, optional) – name of compiler to use
• verbose (bool, optional) – Whether to print extra messages during compilation

Example

The following populates the directory ./model with source and header files:

model.compile(dirpath='./my/model', params={}, verbose=True)

If parallel compilation is enabled (parameter parallel_comp), the files are in the form of ./my/model/header.h, ./my/model/main.c, ./my/model/tu0.c, ./my/model/tu1.c and so forth, depending on the value of parallel_comp. Otherwise, there will be exactly two files: ./model/header.h, ./my/model/main.c

export_as_xgboost(filename, name_obj)

(EXPERIMENTAL FEATURE) Export a tree ensemble model as a XGBoost model file

Parameters:

• filename (str) – path to model file
• name_obj (str) – name of objective function; see objective parameter in the page http://xgboost.readthedocs.io/en/latest/parameter.html#learning-task-parameters

Example

model.export_as_xgboost('xgboost_model.model')

export_lib(toolchain, libpath, params=None, compiler='ast_native', verbose=False, nthread=None, options=None)

Convenience function: Generate prediction code and immediately turn it into a dynamic shared library. A temporary directory will be created to hold the source files.

Parameters:

• toolchain (str) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’. You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)
• libpath (str) – location to save the generated dynamic shared library
• params (dict, optional) – parameters to be passed to the compiler. See this page for the list of compiler parameters.
• compiler (str, optional) – name of compiler to use in C code generation
• verbose (bool, optional) – whether to produce extra messages
• nthread (int, optional) – number of threads to use in creating the shared library. Defaults to the number of cores in the system.
• options (list of str, optional) – Additional options to pass to toolchain

Example

The one-line command

model.export_lib(toolchain='msvc', libpath='./mymodel.dll',
                 params={}, verbose=True)

is equivalent to the following sequence of commands:

model.compile(dirpath='/temporary/directory', params={}, verbose=True)
create_shared(toolchain='msvc', dirpath='/temporary/directory',
              verbose=True)
# move the library out of the temporary directory
shutil.move('/temporary/directory/mymodel.dll', './mymodel.dll')

export_srcpkg(platform, toolchain, pkgpath, libname, params=None, compiler='ast_native', verbose=False, options=None)

Convenience function: Generate prediction code and create a zipped source package for deployment. The resulting zip file will also contain a Makefile.

Parameters:

• platform (str) – name of the operating system on which the headers and sources shall be compiled. Must be one of the following: ‘windows’ (Microsoft Windows), ‘osx’ (Mac OS X), ‘unix’ (Linux and other UNIX-like systems)
• toolchain (str) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’. You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)
• pkgpath (str) – location to save the zipped source package
• libname (str) – name of model shared library to be built
• params (dict, optional) – parameters to be passed to the compiler. See this page for the list of compiler parameters.
• compiler (str, optional) – name of compiler to use in C code generation
• verbose (bool, optional) – whether to produce extra messages
• nthread (int, optional) – number of threads to use in creating the shared library. Defaults to the number of cores in the system.
• options (list of str, optional) – Additional options to pass to toolchain

Example

The one-line command

model.export_srcpkg(platform='unix', toolchain='gcc',
                    pkgpath='./mymodel_pkg.zip', libname='mymodel.so',
                    params={}, verbose=True)

is equivalent to the following sequence of commands:

model.compile(dirpath='/temporary/directory/mymodel',
              params={}, verbose=True)
generate_makefile(dirpath='/temporary/directory/mymodel',
                  platform='unix', toolchain='gcc')
# zip the directory containing C code and Makefile
shutil.make_archive(base_name=pkgpath, format='zip',
                    root_dir='/temporary/directory',
                    base_dir='mymodel/')

classmethod from_xgboost(booster)

Load a tree ensemble model from an XGBoost Booster object

Parameters:booster (object of type xgboost.Booster) – Python handle to XGBoost model Returns:model – loaded model Return type:Model object

Example

bst = xgboost.train(params, dtrain, 10, [(dtrain, 'train')])
xgb_model = Model.from_xgboost(bst)

classmethod load(filename, model_format)

Load a tree ensemble model from a file

Parameters:

• filename (str) – path to model file
• model_format (str) – model file format. Must be one or ‘xgboost’, ‘lightgbm’, ‘protobuf’

Returns:

model – loaded model

Return type:

Model object

Example

xgb_model = Model.load('xgboost_model.model', 'xgboost')

class treelite.ModelBuilder(num_feature, num_output_group=1, random_forest=False, **kwargs)

Builder class for tree ensemble model: provides tools to iteratively build an ensemble of decision trees

Parameters:

• num_feature (int) – number of features used in model being built. We assume that all feature indices are between 0 and (num_feature - 1)
• num_output_group (int, optional) – number of output groups; >1 indicates multiclass classification
• random_forest (bool, optional) – whether the model is a random forest; True indicates a random forest and False indicates gradient boosted trees
• **kwargs – model parameters, to be used to specify the resulting model. Refer to this page for the full list of model parameters.

class Node

Handle to a node in a tree

set_categorical_test_node(feature_id, left_categories, default_left, left_child_key, right_child_key)

Set the node as a test node with categorical split. A list defines all categories that would be classified as the left side. Categories are integers ranging from 0 to n-1, where n is the number of categories in that particular feature.

Parameters:

• feature_id (int) – feature index
• left_categories (list of int) – list of categories belonging to the left child.
• default_left (bool) – default direction for missing values (True for left; False for right)
• left_child_key (int) – unique integer key to identify the left child node
• right_child_key (int) – unique integer key to identify the right child node

set_leaf_node(leaf_value)

Set the node as a leaf node

Parameters:leaf_value (float / list of float) – Usually a single leaf value (weight) of the leaf node. For multiclass random forest classifier, leaf_value should be a list of leaf weights.

set_numerical_test_node(feature_id, opname, threshold, default_left, left_child_key, right_child_key)

Set the node as a test node with numerical split. The test is in the form [feature value] OP [threshold]. Depending on the result of the test, either left or right child would be taken.

Parameters:

• feature_id (int) – feature index
• opname (str) – binary operator to use in the test
• threshold (float) – threshold value
• default_left (bool) – default direction for missing values (True for left; False for right)
• left_child_key (int) – unique integer key to identify the left child node
• right_child_key (int) – unique integer key to identify the right child node

set_root()

Set the node as the root

class Tree

Handle to a decision tree in a tree ensemble Builder

append(tree)

Add a tree at the end of the ensemble

Parameters:tree (Tree object) – tree to be added

Example

builder = ModelBuilder(num_feature=4227)
tree = ...               # build tree somehow
builder.append(tree)     # add tree at the end of the ensemble

commit()

Finalize the ensemble model

Returns:model – finished model Return type:Model object

Example

builder = ModelBuilder(num_feature=4227)
for i in range(100):
  tree = ...                    # build tree somehow
  builder.append(tree)          # add one tree at a time

model = builder.commit()        # now get a Model object
model.compile(dirpath='test')   # compile model into C code

insert(tree, index)

Insert a tree at specified location in the ensemble

Parameters:

• tree (Tree object) – tree to be inserted
• index (int) – index of the element before which to insert the tree

Example

builder = ModelBuilder(num_feature=4227)
tree = ...               # build tree somehow
builder.insert(tree, 0)  # insert tree at index 0

class treelite.Annotator(path=None)

Branch annotator class: annotate branches in a given model using frequency patterns in the training data

Parameters:path (str, optional) – if given, the predictor will load branch frequency information from the path

annotate_branch(model, dmat, nthread=None, verbose=False)

Annotate branches in a given model using frequency patterns in the training data. Each node gets the count of the instances that belong to it. Any prior annotation information stored in the annotator will be replaced with the new annotation returned by this method.

Parameters:

• model (object of type Model) – decision tree ensemble model
• dmat (object of type DMatrix) – data matrix representing the training data
• nthread (int, optional) – number of threads to use while annotating. If missing, use all physical cores in the system.
• verbose (bool, optional) – whether to produce extra messages

save(path)

Save branch annotation infromation as a JSON file.

Parameters:path (str) – location of saved JSON file

treelite.create_shared(toolchain, dirpath, nthread=None, verbose=False, options=None)

Create shared library.

Parameters:

• toolchain (str) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’. You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)
• dirpath (str) – directory containing the header and source files previously generated by Model.compile(). The directory must contain recipe.json which specifies build dependencies.
• nthread (int, optional) – number of threads to use in creating the shared library. Defaults to the number of cores in the system.
• verbose (bool, optional) – whether to produce extra messages
• options (list of str, optional) – Additional options to pass to toolchain

Returns:

libpath – absolute path of created shared library

Return type:

str

Example

The following command uses Visual C++ toolchain to generate ./my/model/model.dll:

model.compile(dirpath='./my/model', params={}, verbose=True)
create_shared(toolchain='msvc', dirpath='./my/model', verbose=True)

Later, the shared library can be referred to by its directory name:

predictor = Predictor(libpath='./my/model', verbose=True)
# looks for ./my/model/model.dll

Alternatively, one may specify the library down to its file name:

predictor = Predictor(libpath='./my/model/model.dll', verbose=True)

treelite.save_runtime_package(destdir, include_binary=False)

Save a copy of the (zipped) runtime package, containing all glue code necessary to deploy compiled models into the wild

Parameters:

• destdir (str) – directory to save the zipped package
• include_binary (boolean, optional (defaults to False)) – whether to include the compiled binary (.dll/.so/.dylib) in the package. If this option is enabled, you won’t have to run the CMake script on the target machine (the machine to run the deployed model). Warning: To enable this option, the host machine (one running treelite) must use the same platform (OS + CPU) as the target machine. This is to ensure binary compatibility. If the host and target machines differ in platform, they cannot share the compiled binary.

treelite.generate_makefile(dirpath, platform, toolchain, options=None)

Generate a Makefile for a given directory of headers and sources. The resulting Makefile will be stored in the directory. This function is useful for deploying a model on a different machine.

Parameters:

• dirpath (str) – directory containing the header and source files previously generated by Model.compile(). The directory must contain recipe.json which specifies build dependencies.
• platform (str) – name of the operating system on which the headers and sources shall be compiled. Must be one of the following: ‘windows’ (Microsoft Windows), ‘osx’ (Mac OS X), ‘unix’ (Linux and other UNIX-like systems)
• toolchain (str) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’. You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)
• options (list of str, optional) – Additional options to pass to toolchain

---

treelite.gallery.sklearn.import_model(sklearn_model)

Load a tree ensemble model from a scikit-learn model object

Parameters:sklearn_model (object of type RandomForestRegressor / RandomForestClassifier / GradientBoostingRegressor / GradientBoostingClassifier) – Python handle to scikit-learn model Returns:model – loaded model Return type:Model object

Example

import sklearn.datasets
import sklearn.ensemble
X, y = sklearn.datasets.load_boston(return_X_y=True)
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)

import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)

Treelite runtime API

Runtime API of treelite Python package.

Runtime API provides the minimum necessary tools to deploy tree prediction modules in the wild.

class treelite.runtime.Predictor(libpath, nthread=None, verbose=False, include_master_thread=True)

Predictor class: loader for compiled shared libraries

Parameters:

• libpath (str) – location of dynamic shared library (.dll/.so/.dylib)
• nthread (int, optional) – number of worker threads to use; if unspecified, use maximum number of hardware threads
• verbose (bool, optional) – Whether to print extra messages during construction
• include_master_thread (bool, optional) – Whether to assign work to the master thread

predict(batch, verbose=False, pred_margin=False)

Make prediction using a batch of data rows (synchronously). This will internally split workload among worker threads.

Parameters:

• batch (object of type Batch) – batch of rows for which predictions will be made
• verbose (bool, optional) – Whether to print extra messages during prediction
• pred_margin (bool, optional) – whether to produce raw margins rather than transformed probabilities

class treelite.runtime.Batch

Batch of rows to be used for prediction

classmethod from_csr(csr, rbegin=None, rend=None)

Get a sparse batch from a subset of rows in a CSR (Compressed Sparse Row) matrix. The subset is given by the range [rbegin, rend).

Parameters:

• csr (object of class treelite.DMatrix or scipy.sparse.csr_matrix) – data matrix
• rbegin (int, optional) – the index of the first row in the subset
• rend (int, optional) – one past the index of the last row in the subset. If missing, set to the end of the matrix.

Returns:

sparse_batch – a sparse batch consisting of rows [rbegin, rend)

Return type:

Batch

classmethod from_npy2d(mat, rbegin=0, rend=None, missing=None)

Get a dense batch from a 2D numpy matrix. If mat does not have order='C' (also known as row-major) or is not contiguous, a temporary copy will be made. If mat does not have dtype=numpy.float32, a temporary copy will be made also. Thus, as many as two temporary copies of data can be made. One should set input layout and type judiciously to conserve memory.

Parameters:

• mat (object of type numpy.ndarray, with dimension 2) – data matrix
• rbegin (int, optional) – the index of the first row in the subset
• rend (int, optional) – one past the index of the last row in the subset. If missing, set to the end of the matrix.
• missing (float, optional) – value indicating missing value. If missing, set to numpy.nan.

Returns:

dense_batch – a dense batch consisting of rows [rbegin, rend)

Return type:

Batch

shape()

Get dimensions of the batch

Returns:dims – (number of rows, number of columns) Return type:tuple of length 2

Treelite C API

Treelite exposes a set of C functions to enable interfacing with a variety of languages. This page will be most useful for:

• those writing a new language binding (glue code).
• those wanting to incorporate functions of treelite into their own native libraries.

We recommend the Python API for everyday uses.

Note

Use of C and C++ in treelite

Core logic of treelite are written in C++ to take advantage of higher abstractions. We provide C only interface here, as many more programming languages bind with C than with C++. See this page for more details.

Data matrix interface

Use the following functions to load and manipulate data from a variety of sources.

int TreeliteDMatrixCreateFromFile(const char *path, const char *format, int nthread, int verbose, DMatrixHandle *out)

create DMatrix from a file

Return

0 for success, -1 for failure

Parameters

• path: file path
• format: file format
• nthread: number of threads to use
• verbose: whether to produce extra messages
• out: the created DMatrix

int TreeliteDMatrixCreateFromCSR(const float *data, const unsigned *col_ind, const size_t *row_ptr, size_t num_row, size_t num_col, DMatrixHandle *out)

create DMatrix from a (in-memory) CSR matrix

Return

0 for success, -1 for failure

Parameters

• data: feature values
• col_ind: feature indices
• row_ptr: pointer to row headers
• num_row: number of rows
• num_col: number of columns
• out: the created DMatrix

int TreeliteDMatrixCreateFromMat(const float *data, size_t num_row, size_t num_col, float missing_value, DMatrixHandle *out)

create DMatrix from a (in-memory) dense matrix

Return

0 for success, -1 for failure

Parameters

• data: feature values
• num_row: number of rows
• num_col: number of columns
• missing_value: value to represent missing value
• out: the created DMatrix

int TreeliteDMatrixGetDimension(DMatrixHandle handle, size_t *out_num_row, size_t *out_num_col, size_t *out_nelem)

get dimensions of a DMatrix

Return

0 for success, -1 for failure

Parameters

• handle: handle to DMatrix
• out_num_row: used to set number of rows
• out_num_col: used to set number of columns
• out_nelem: used to set number of nonzero entries

int TreeliteDMatrixGetPreview(DMatrixHandle handle, const char **out_preview)

produce a human-readable preview of a DMatrix Will print first and last 25 non-zero entries, along with their locations

Return

0 for success, -1 for failure

Parameters

• handle: handle to DMatrix
• out_preview: used to save the address of the string literal

int TreeliteDMatrixGetArrays(DMatrixHandle handle, const float **out_data, const uint32_t **out_col_ind, const size_t **out_row_ptr)

extract three arrays (data, col_ind, row_ptr) that define a DMatrix.

Return

0 for success, -1 for failure

Parameters

• handle: handle to DMatrix
• out_data: used to save pointer to array containing feature values
• out_col_ind: used to save pointer to array containing feature indices
• out_row_ptr: used to save pointer to array containing pointers to row headers

int TreeliteDMatrixFree(DMatrixHandle handle)

delete DMatrix from memory

Return

0 for success, -1 for failure

Parameters

• handle: handle to DMatrix

Branch annotator interface

Use the following functions to annotate branches in decision trees.

int TreeliteAnnotateBranch(ModelHandle model, DMatrixHandle dmat, int nthread, int verbose, AnnotationHandle *out)

annotate branches in a given model using frequency patterns in the training data.

Return

0 for success, -1 for failure

Parameters

• model: model to annotate
• dmat: training data matrix
• nthread: number of threads to use
• verbose: whether to produce extra messages
• out: used to save handle for the created annotation

int TreeliteAnnotationLoad(const char *path, AnnotationHandle *out)

load branch annotation from a JSON file

Return

0 for success, -1 for failure

Parameters

• path: path to JSON file
• out: used to save handle for the loaded annotation

int TreeliteAnnotationSave(AnnotationHandle handle, const char *path)

save branch annotation to a JSON file

Return

0 for success, -1 for failure

Parameters

• handle: annotation to save
• path: path to JSON file

int TreeliteAnnotationFree(AnnotationHandle handle)

delete branch annotation from memory

Return

0 for success, -1 for failure

Parameters

• handle: annotation to remove

Compiler interface

Use the following functions to produce optimize prediction subroutine (in C) from a given decision tree ensemble.

int TreeliteCompilerCreate(const char *name, CompilerHandle *out)

create a compiler with a given name

Return

0 for success, -1 for failure

Parameters

• name: name of compiler
• out: created compiler

int TreeliteCompilerSetParam(CompilerHandle handle, const char *name, const char *value)

set a parameter for a compiler

Return

0 for success, -1 for failure

Parameters

• handle: compiler
• name: name of parameter
• value: value of parameter

int TreeliteCompilerGenerateCode(CompilerHandle compiler, ModelHandle model, int verbose, const char *dirpath)

generate prediction code from a tree ensemble model. The code will be C99 compliant. One header file (.h) will be generated, along with one or more source files (.c).

Usage example:

TreeliteCompilerGenerateCode(compiler, model, 1, "./my/model");
// files to generate: ./my/model/header.h, ./my/model/main.c
// if parallel compilation is enabled:
// ./my/model/header.h, ./my/model/main.c, ./my/model/tu0.c,
// ./my/model/tu1.c, and so forth

Return

0 for success, -1 for failure

Parameters

• compiler: handle for compiler
• model: handle for tree ensemble model
• verbose: whether to produce extra messages
• dirpath: directory to store header and source files

int TreeliteCompilerFree(CompilerHandle handle)

delete compiler from memory

Return

0 for success, -1 for failure

Parameters

• handle: compiler to remove

Model loader interface

Use the following functions to load decision tree ensemble models from a file. Treelite supports multiple model file formats.

int TreeliteLoadLightGBMModel(const char *filename, ModelHandle *out)

load a model file generated by LightGBM (Microsoft/LightGBM). The model file must contain a decision tree ensemble.

Return

0 for success, -1 for failure

Parameters

• filename: name of model file
• out: loaded model

int TreeliteLoadXGBoostModel(const char *filename, ModelHandle *out)

load a model file generated by XGBoost (dmlc/xgboost). The model file must contain a decision tree ensemble.

Return

0 for success, -1 for failure

Parameters

• filename: name of model file
• out: loaded model

int TreeliteLoadXGBoostModelFromMemoryBuffer(const void *buf, size_t len, ModelHandle *out)

load an XGBoost model from a memory buffer.

Return

0 for success, -1 for failure

Parameters

• buf: memory buffer
• len: size of memory buffer
• out: loaded model

int TreeliteLoadProtobufModel(const char *filename, ModelHandle *out)

load a model in Protocol Buffers format. Protocol Buffers (google/protobuf) is a language- and platform-neutral mechanism for serializing structured data. See tree.proto for format spec.

Return

0 for success, -1 for failure

Parameters

• filename: name of model file
• out: loaded model

int TreeliteExportXGBoostModel(const char *filename, ModelHandle model, const char *name_obj)

(EXPERIMENTAL FEATURE) export a model in XGBoost format. The exported model can be read by XGBoost (dmlc/xgboost).

Return

0 for success, -1 for failure

Parameters

• filename: name of model file
• model: model to export
• name_obj: name of objective function

int TreeliteFreeModel(ModelHandle handle)

delete model from memory

Return

0 for success, -1 for failure

Parameters

• handle: model to remove

Model builder interface

Use the following functions to incrementally build decisio n tree ensemble models.

int TreeliteCreateTreeBuilder(TreeBuilderHandle *out)

Create a new tree builder.

Return

0 for success; -1 for failure

Parameters

• out: newly created tree builder

int TreeliteDeleteTreeBuilder(TreeBuilderHandle handle)

Delete a tree builder from memory.

Return

0 for success; -1 for failure

Parameters

• handle: tree builder to remove

int TreeliteTreeBuilderCreateNode(TreeBuilderHandle handle, int node_key)

Create an empty node within a tree.

Return

0 for success; -1 for failure

Parameters

• handle: tree builder
• node_key: unique integer key to identify the new node

int TreeliteTreeBuilderDeleteNode(TreeBuilderHandle handle, int node_key)

Remove a node from a tree.

Return

0 for success; -1 for failure

Parameters

• handle: tree builder
• node_key: unique integer key to identify the node to be removed

int TreeliteTreeBuilderSetRootNode(TreeBuilderHandle handle, int node_key)

Set a node as the root of a tree.

Return

0 for success; -1 for failure

Parameters

• handle: tree builder
• node_key: unique integer key to identify the root node

int TreeliteTreeBuilderSetNumericalTestNode(TreeBuilderHandle handle, int node_key, unsigned feature_id, const char *opname, float threshold, int default_left, int left_child_key, int right_child_key)

Turn an empty node into a test node with numerical split. The test is in the form [feature value] OP [threshold]. Depending on the result of the test, either left or right child would be taken.

Return

0 for success; -1 for failure

Parameters

• handle: tree builder
• node_key: unique integer key to identify the node being modified; this node needs to be empty
• feature_id: id of feature
• opname: binary operator to use in the test
• threshold: threshold value
• default_left: default direction for missing values
• left_child_key: unique integer key to identify the left child node
• right_child_key: unique integer key to identify the right child node

int TreeliteTreeBuilderSetCategoricalTestNode(TreeBuilderHandle handle, int node_key, unsigned feature_id, const unsigned int *left_categories, size_t left_categories_len, int default_left, int left_child_key, int right_child_key)

Turn an empty node into a test node with categorical split. A list defines all categories that would be classified as the left side. Categories are integers ranging from 0 to (n-1), where n is the number of categories in that particular feature. Let’s assume n <= 64.

Return

0 for success; -1 for failure

Parameters

• handle: tree builder
• node_key: unique integer key to identify the node being modified; this node needs to be empty
• feature_id: id of feature
• left_categories: list of categories belonging to the left child
• left_categories_len: length of left_cateogries
• default_left: default direction for missing values
• left_child_key: unique integer key to identify the left child node
• right_child_key: unique integer key to identify the right child node

int TreeliteTreeBuilderSetLeafNode(TreeBuilderHandle handle, int node_key, float leaf_value)

Turn an empty node into a leaf node.

Return

0 for success; -1 for failure

Parameters

• handle: tree builder
• node_key: unique integer key to identify the node being modified; this node needs to be empty
• leaf_value: leaf value (weight) of the leaf node

int TreeliteTreeBuilderSetLeafVectorNode(TreeBuilderHandle handle, int node_key, const float *leaf_vector, size_t leaf_vector_len)

Turn an empty node into a leaf vector node The leaf vector (collection of multiple leaf weights per leaf node) is useful for multi-class random forest classifier.

Return

0 for success; -1 for failure

Parameters

• handle: tree builder
• node_key: unique integer key to identify the node being modified; this node needs to be empty
• leaf_vector: leaf vector of the leaf node
• leaf_vector_len: length of leaf_vector

int TreeliteCreateModelBuilder(int num_feature, int num_output_group, int random_forest_flag, ModelBuilderHandle *out)

Create a new model builder.

Return

0 for success; -1 for failure

Parameters

• num_feature: number of features used in model being built. We assume that all feature indices are between 0 and (num_feature - 1).
• num_output_group: number of output groups. Set to 1 for binary classification and regression; >1 for multiclass classification
• random_forest_flag: whether the model is a random forest. Set to 0 if the model is gradient boosted trees. Any nonzero value shall indicate that the model is a random forest.
• out: newly created model builder

int TreeliteModelBuilderSetModelParam(ModelBuilderHandle handle, const char *name, const char *value)

Set a model parameter.

Return

0 for success; -1 for failure

Parameters

• handle: model builder
• name: name of parameter
• value: value of parameter

int TreeliteDeleteModelBuilder(ModelBuilderHandle handle)

Delete a model builder from memory.

Return

0 for success; -1 for failure

Parameters

• handle: model builder to remove

int TreeliteModelBuilderInsertTree(ModelBuilderHandle handle, TreeBuilderHandle tree_builder, int index)

Insert a tree at specified location.

Return

index of the new tree within the ensemble; -1 for failure

Parameters

• handle: model builder
• tree_builder: builder for the tree to be inserted. The tree must not be part of any other existing tree ensemble. Note: The tree_builder argument will become unusuable after the tree insertion. Should you want to modify the tree afterwards, use GetTree(*) method to get a fresh handle to the tree.
• index: index of the element before which to insert the tree; use -1 to insert at the end

int TreeliteModelBuilderGetTree(ModelBuilderHandle handle, int index, TreeBuilderHandle *out)

Get a reference to a tree in the ensemble.

Return

0 for success; -1 for failure

Parameters

• handle: model builder
• index: index of the tree in the ensemble
• out: used to save reference to the tree

int TreeliteModelBuilderDeleteTree(ModelBuilderHandle handle, int index)

Remove a tree from the ensemble.

Return

0 for success; -1 for failure

Parameters

• handle: model builder
• index: index of the tree that would be removed

int TreeliteModelBuilderCommitModel(ModelBuilderHandle handle, ModelHandle *out)

finalize the model and produce the in-memory representation

Return

0 for success; -1 for failure

Parameters

• handle: model builder
• out: used to save handle to in-memory representation of the finished model

Predictor interface

Use the following functions to load compiled prediction subroutines from shared libraries and to make predictions.

int TreeliteAssembleSparseBatch(const float *data, const uint32_t *col_ind, const size_t *row_ptr, size_t num_row, size_t num_col, CSRBatchHandle *out)

assemble a sparse batch

Return

0 for success, -1 for failure

Parameters

• data: feature values
• col_ind: feature indices
• row_ptr: pointer to row headers
• num_row: number of data rows in the batch
• num_col: number of columns (features) in the batch
• out: handle to sparse batch

int TreeliteDeleteSparseBatch(CSRBatchHandle handle)

delete a sparse batch from memory

Return

0 for success, -1 for failure

Parameters

• handle: sparse batch

int TreeliteAssembleDenseBatch(const float *data, float missing_value, size_t num_row, size_t num_col, DenseBatchHandle *out)

assemble a dense batch

Return

0 for success, -1 for failure

Parameters

• data: feature values
• missing_value: value to represent the missing value
• num_row: number of data rows in the batch
• num_col: number of columns (features) in the batch
• out: handle to sparse batch

int TreeliteDeleteDenseBatch(DenseBatchHandle handle)

delete a dense batch from memory

Return

0 for success, -1 for failure

Parameters

• handle: dense batch

int TreeliteBatchGetDimension(void *handle, int batch_sparse, size_t *out_num_row, size_t *out_num_col)

get dimensions of a batch

Return

0 for success, -1 for failure

Parameters

• handle: a batch of rows (must be of type SparseBatch or DenseBatch)
• batch_sparse: whether the batch is sparse (true) or dense (false)
• out_num_row: used to set number of rows
• out_num_col: used to set number of columns

int TreelitePredictorLoad(const char *library_path, int num_worker_thread, int include_master_thread, PredictorHandle *out)

load prediction code into memory. This function assumes that the prediction code has been already compiled into a dynamic shared library object (.so/.dll/.dylib).

Return

0 for success, -1 for failure

Parameters

• library_path: path to library object file containing prediction code
• num_worker_thread: number of worker threads (-1 to use max number)
• include_master_thread: whether to assign workload to the master thread. If not, only workers threads will be assigned work.
• out: handle to predictor

int TreelitePredictorPredictBatch(PredictorHandle handle, void *batch, int batch_sparse, int verbose, int pred_margin, float *out_result, size_t *out_result_size)

Make predictions on a batch of data rows (synchronously). This function internally divides the workload among all worker threads.

Return

0 for success, -1 for failure

Parameters

• handle: predictor
• batch: a batch of rows (must be of type SparseBatch or DenseBatch)
• batch_sparse: whether batch is sparse (1) or dense (0)
• verbose: whether to produce extra messages
• pred_margin: whether to produce raw margin scores instead of transformed probabilities
• out_result: resulting output vector; use TreelitePredictorQueryResultSize() to allocate sufficient space
• out_result_size: used to save length of the output vector, which is guaranteed to be less than or equal to TreelitePredictorQueryResultSize()

int TreelitePredictorQueryResultSize(PredictorHandle handle, void *batch, int batch_sparse, size_t *out)

Given a batch of data rows, query the necessary size of array to hold predictions for all data points.

Return

0 for success, -1 for failure

Parameters

• handle: predictor
• batch: a batch of rows (must be of type SparseBatch or DenseBatch)
• batch_sparse: whether batch is sparse (1) or dense (0)
• out: used to store the length of prediction array

int TreelitePredictorQueryNumOutputGroup(PredictorHandle handle, size_t *out)

Get the number of output groups in the loaded model The number is 1 for most tasks; it is greater than 1 for multiclass classifcation.

Return

0 for success, -1 for failure

Parameters

• handle: predictor
• out: length of prediction array

int TreelitePredictorFree(PredictorHandle handle)

delete predictor from memory

Return

0 for success, -1 for failure

Parameters

• handle: predictor to remove

Handle types

Treelite uses C++ classes to define its internal data structures. In order to pass C++ objects to C functions, opaque handles are used. Opaque handles are void* pointers that store raw memory addresses.

typedef void *DMatrixHandle

handle to a data matrix

typedef void *ModelHandle

handle to a decision tree ensemble model

typedef void *TreeBuilderHandle

handle to tree builder class

typedef void *ModelBuilderHandle

handle to ensemble builder class

typedef void *AnnotationHandle

handle to branch annotation data

typedef void *CompilerHandle

handle to compiler class

typedef void *PredictorHandle

handle to predictor class

typedef void *CSRBatchHandle

handle to batch of sparse data rows

typedef void *DenseBatchHandle

handle to batch of dense data rows

Knobs and Parameters

Compiler Parameters

Compiler parameters influence the way the prediction subroutine is generated from a tree ensemble model.

---

std::string annotate_in

name of model annotation file. Use the class treelite.Annotator to generate this file.

int quantize

whether to quantize threshold points (0: no, >0: yes)

int parallel_comp

option to enable parallel compilation; if set to nonzero, the trees will be evely distributed into [parallel_comp] files. Set this option to improve compilation time and reduce memory consumption during compilation.

int verbose

if >0, produce extra messages

Model Parameters

Model parameters are used by ModelBuilder class to clarify certain behaviors of a tree ensemble model.

---

std::string pred_transform

name of prediction transform function

This parameter specifies how to transform raw margin values into final predictions. By default, this is set to 'identity', which means no transformation.

For the multi-class classification task, pred_transfrom must be one of the following values:

 - identity_multiclass
   do not transform. The output will be a matrix with dimensions
   [number of data points] * [number of classes] that contains the margin score
   for every (data point, class) pair.
 - max_index
   compute the most probable class for each data point and output the class
   index. The output will be a vector of length [number of data points] that
   contains the most likely class of each data point.
 - softmax
   use the softmax function to transform a multi-dimensional vector into a
   proper probability distribution. The output will be a matrix with dimensions
   [number of data points] * [number of classes] that contains the predicted
   probability of each data point belonging to each class.
 - multiclass_ova
   apply the sigmoid function element-wise to the margin matrix. The output will
   be a matrix with dimensions [number of data points] * [number of classes].

For all other tasks (e.g. regression, binary classification, ranking etc.), pred_transfrom must be one of the following values:

  - identity
    do not transform. The output will be a vector of length
    [number of data points] that contains the margin score for every data point.
  - sigmoid
    apply the sigmoid function element-wise to the margin vector. The output
    will be a vector of length [number of data points] that contains the
    probability of each data point belonging to the positive class.
  - exponential
    apply the exponential function (exp) element-wise to the margin vector. The
    output will be a vector of length [number of data points].
  - logarithm_one_plus_exp
    apply the function f(x) = log(1 + exp(x)) element-wise to the margin vector.
    The output will be a vector of length [number of data points].

float sigmoid_alpha

scaling parameter for sigmoid function sigmoid(x) = 1 / (1 + exp(-alpha * x))

This parameter is used only when pred_transform is set to 'sigmoid'. It must be strictly positive; if unspecified, it is set to 1.0.

float global_bias

global bias of the model

Predicted margin scores of all instances will be adjusted by the global bias. If unspecified, the bias is set to zero.

Indices

• Index
• Module Index