Treelite is a model compiler for decision tree ensembles, aimed at efficient deployment.
Star WatchYou are currently browsing the documentation of a stable version of Treelite: 1.3.0.
Treelite compiles your tree model into optimized shared library. A Benchmark demonstrates 2-6x improvement in prediction throughput, due to more efficient use of compute resources.
Treelite accommodates a wide range of decision tree ensemble models. In particular, it handles both random forests and gradient boosted trees.
Treelite can read models produced by XGBoost, LightGBM, and scikit-learn. In cases where you are using another package to train your model, you may use the flexible builder class.
It is a great hassle to install machine learning packages (e.g. XGBoost, LightGBM, scikit-learn, etc.) on every machine your tree model will run. This is the case no longer: Treelite will export your model as a stand-alone prediction library so that predictions will be made without any machine learning package installed.
Install Treelite from PyPI:
python3 -m pip install --user treelite treelite_runtime
Import your tree ensemble model into Treelite:
import treelite
model = treelite.Model.load('my_model.model', model_format='xgboost')
Deploy a source archive:
# Produce a zipped source directory, containing all model information
# Run `make` on the target machine
model.export_srcpkg(platform='unix', toolchain='gcc',
pkgpath='./mymodel.zip', libname='mymodel.so',
verbose=True)
Deploy a shared library:
# Like export_srcpkg, but generates a shared library immediately
# Use this only when the host and target machines are compatible
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True)
Make predictions on the target machine:
import treelite_runtime
predictor = treelite_runtime.Predictor('./mymodel.so', verbose=True)
dmat = treelite_runtime.DMatrix(X)
out_pred = predictor.predict(dmat)
Read First tutorial for a more detailed example. See Deploying models for additional instructions on deployment.
Note
A note on API compatibility
Since Treelite is in early development, its API may change substantially in the future.
See the page Benchmark for details.
The workflow involves two distinct machines: the host machine that generates prediction subroutine from a given tree model, and the target machine that runs the subroutine. The two machines exchange a single C file that contains all relevant information about the tree model. Only the host machine needs to have Treelite installed; the target machine requires only a working C compiler.
You may choose one of two methods to install Treelite on your system:
This is probably the most convenient method. Simply type
python3 -m pip install --user treelite treelite_runtime
to install the Treelite package. The command will locate the binary release that is compatible with your current platform. Check the installation by running
import treelite
import treelite_runtime
in an interactive Python session. This method is available for only Windows, Mac OS X, and Linux. For other operating systems, see the next section.
Note
Installing OpenMP runtime on Mac OSX
Treelite requires the presence of OpenMP runtime. To install OpenMP runtime on a Mac OSX system, run the following command:
brew install libomp
Installation consists of two steps:
Build the shared libraries from C++ code (See the note below for the list.)
Install the Python package.
Note
List of libraries created
There will be two libraries created: the main library, for producing optimized prediction subroutines; and the runtime library, for deploying these subroutines in the wild for actual prediction tasks.
Operating System |
Main library |
Runtime library |
---|---|---|
Windows |
|
|
Mac OS X |
|
|
Linux / other UNIX |
|
|
To get started, clone Treelite repo from GitHub.
git clone https://github.com/dmlc/treelite.git
cd treelite
The next step is to build the shared libraries.
The Python package is located at the python
subdirectory. There are several
ways to install the package:
1. Install system-wide, which requires root permission
# Install treelite
cd python
sudo python3 setup.py install
# Install treelite_runtime
cd ../runtime/python
sudo python3 setup.py install
You will need Python setuptools
module for this to work. It is often part of the core Python installation.
Should it be necessary, the package can be installed using pip
:
pip install -U pip setuptools
2. Install for only current user
This is useful if you do not have the administrative rights.
# Install treelite
cd python
python3 setup.py install --user
# Install treelite_runtime
cd ../runtime/python
python3 setup.py install --user
Note
Recompiling Treelite
Every time the C++ portion of Treelite gets re-compiled, the Python package must be re-installed for the new library to take effect.
3. Set the environment variable PYTHONPATH to locate Treelite package
Only set the environment variable PYTHONPATH
to tell Python where to find
the Treelite package. This is useful for developers, as any changes made
to C++ code will be immediately visible to Python side without re-running setup.py
.
export PYTHONPATH=/path/to/treelite/python:/path/to/treelite/runtime/python
python3 # enter interactive session
This page lists tutorials about Treelite.
This tutorial will demonstrate the basic workflow.
import treelite
In this tutorial, we will use a small regression example to describe the full workflow.
Let us use the Boston house prices dataset from scikit-learn
(sklearn.datasets.load_boston()
). It consists of 506 houses
with 13 distinct features:
from sklearn.datasets import load_boston
X, y = load_boston(return_X_y=True)
print(f'dimensions of X = {X.shape}')
print(f'dimensions of y = {y.shape}')
The first step is to train a tree ensemble model using XGBoost (dmlc/xgboost).
Disclaimer: Treelite does NOT depend on the XGBoost package in any way. XGBoost was used here only to provide a working example.
import xgboost
dtrain = xgboost.DMatrix(X, label=y)
params = {'max_depth':3, 'eta':1, 'objective':'reg:squarederror', 'eval_metric':'rmse'}
bst = xgboost.train(params, dtrain, 20, [(dtrain, 'train')])
Next, we feed the trained model into Treelite. If you used XGBoost to train the model, it takes only one line of code:
model = treelite.Model.from_xgboost(bst)
Note
Using other packages to train decision trees
With additional work, you can use models trained with other machine learning packages. See this page for instructions.
Since the scope of Treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.
Contents
XGBoost (dmlc/xgboost) is a fast, scalable package for gradient boosting. Both Treelite and XGBoost are hosted by the DMLC (Distributed Machine Learning Community) group.
Treelite plays well with XGBoost — if you used XGBoost to train your ensemble model, you need only one line of code to import it. Depending on where your model is located, you should do one of the following:
Load XGBoost model from a xgboost.Booster
object
# bst = an object of type xgboost.Booster
model = Model.from_xgboost(bst)
Load XGBoost model from a binary model file
# model had been saved to a file named my_model.model
# notice the second argument model_format='xgboost'
model = Model.load('my_model.model', model_format='xgboost')
LightGBM (Microsoft/LightGBM) is
another well known machine learning package for gradient boosting. To import
models generated by LightGBM, use the load()
method
with argument model_format='lightgbm'
:
# model had been saved to a file named my_model.txt
# notice the second argument model_format='lightgbm'
model = Model.load('my_model.txt', model_format='lightgbm')
Scikit-learn (scikit-learn/scikit-learn) is a Python machine learning package known for its versatility and ease of use. It supports a wide variety of models and algorithms. The following kinds of models can be imported into Treelite.
To import scikit-learn models, use
treelite.sklearn.import_model()
:
# clf is the model object generated by scikit-learn
import treelite.sklearn
model = treelite.sklearn.import_model(clf)
If you used other packages to train your ensemble model, you’d need to specify the model programmatically:
Treelite offers system-level optimizations to boost prediction performance. Notice that the model information is wholly preserved; the optimizations only affect the manner at which prediction is performed.
Contents
This optimization analyzes and annotates every threshold conditions in the test nodes to improve performance.
The first step is to generate the branch annotation record for your ensemble model. Make sure to have your training data ready.
# model = your ensemble model (object of type treelite.Model)
# dmat = training data (object of type treelite.DMatrix)
# Create annotator object
annotator = treelite.Annotator()
# Annotate branches by iterating over the training data
annotator.annotate_branch(model=model, dmat=dmat, verbose=True)
# Save the branch annotation record as a JSON file
annotator.save(path='mymodel-annotation.json')
To utilize the branch annotation record, supply the compiler parameter
annotate_in
when exporting the model:
# Export a source directory
model.compile(dirpath='./mymodel', verbose=True,
params={'annotate_in': 'mymodel-annotation.json'})
# Export a source directory, packaged in a zip archive
model.export_srcpkg(platform='unix', toolchain='gcc', pkgpath='./mymodel.zip',
libname='mymodel.so', verbose=True,
params={'annotate_in': 'mymodel-annotation.json'})
# Export a shared library
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True,
params={'annotate_in': 'mymodel-annotation.json'})
Modern CPUs heavily rely on a technique known as
branch prediction, in which
they “guess” the result of the conditional expression in each if
-else
branch ahead of time. Given a program
if ( [conditional expression] ) {
foo();
} else {
bar();
}
the CPU will pre-fetch the instructions for the function foo()
if the given
condition is likely to be true. On the other hand, if the condition is likely
to be false, the CPU will pre-fetch the instructions for the function bar()
.
It suffices to say that correctly predicting conditional branches has
great impact on performance. Each time the CPU predicts a branch correctly, it
can keep the instructions it had pre-fetched earlier. Each time the CPU fails to
predict, it must throw away the pre-fetched instructions and fetch anew another
set of instructions. If you’d like to learn more about the importance of branch
prediction, read
this excellent introductory article
from Stanford.
The prediction subroutine for a decision tree ensemble is problematic, as it is replete with conditional branches that will have to be guessed well:
/* A slice of prediction subroutine */
float predict_margin(const float* data) {
float sum = 0.0f;
if (!(data[0].missing != -1) || data[0].fvalue <= 9.5) {
if (!(data[0].missing != -1) || data[0].fvalue <= 3.5) {
if (!(data[10].missing != -1) || data[10].fvalue <= 0.74185) {
if (!(data[0].missing != -1) || data[0].fvalue <= 1.5) {
if (!(data[2].missing != -1) || data[2].fvalue <= 2.08671) {
if ( (data[4].missing != -1) && data[4].fvalue <= 2.02632) {
if (!(data[3].missing != -1) || data[3].fvalue <= 0.763339) {
sum += (float)0.00758165;
} else {
sum += (float)0.0060202;
}
} else {
if ( (data[1].missing != -1) && data[1].fvalue <= 0.0397456) {
sum += (float)0.00415399;
} else {
sum += (float)0.00821985;
}
}
/* and so forth... */
In fact, each threshold condition in the test nodes will need to be predicted. While CPUs lack adequate information to make good guesses on these conditions, we can help by providing that information.
We predict the likelihood of each condition by counting the number of data points from the training data that satisfy that condition. See the diagram below for an illustration.
If a condition is true at least 50% of the time (over the training data), the condition is labeled as “expected to be true”:
/* expected to be true */
if ( __builtin_expect( [condition], 1 ) ) {
...
} else {
...
}
On the other hand, if a condition is false at least 50% of the time, the condition is labeled as “expected to be false”:
/* expected to be false */
if ( __builtin_expect( [condition], 0 ) ) {
...
} else {
...
}
Note
On the expression __builtin_expect
The __builtin_expect
expression is a compiler intrinsic to supply the C
compiler with branch prediction information. Both
gcc
and clang
support it. Unfortunately, Microsoft Visual C++ does not. To take advantage
of branch annotation, make sure to use gcc or clang on the target machine.
This optimization replaces all thresholds in the test nodes with integers so that each threshold condition performs integer comparison instead of the usual floating-point comparison. The thresholds are said to be quantized into integer indices.
BEFORE:
if (data[3].fvalue < 1.5) { /* floating-point comparison */
...
}
AFTER:
if (data[3].qvalue < 3) { /* integer comparison */
...
}
Simply add the compiler parameter quantize=1
when exporting the model:
# Export a source directory
model.compile(dirpath='./mymodel', verbose=True,
params={'quantize': 1})
# Export a source directory, packaged in a zip archive
model.export_srcpkg(platform='unix', toolchain='gcc', pkgpath='./mymodel.zip',
libname='mymodel.so', verbose=True,
params={'quantize': 1})
# Export a shared library
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True,
params={'quantize': 1})
On some platforms such as x86-64, replacing floating-point thresholds with integers helps improve performance by 1) reducing executable code size and 2) improving data locality. This is so because on these platforms, integer constants can be embedded as part of the comparison instruction, whereas floating-point constants cannot.
Let’s look at x86-64 platform. The integer comparison
a <= 4
produces one assembly instruction:
cmpl $4, 8(%rsp) ; 8(%rsp) contains the variable a
Since the integer constant 4
got embedded into the comparison instruction
cmpl, we only
had to fetch the variable a
from memory.
On the other hand, the floating-point comparison
b < 1.2f
produces two assembly instructions:
movss 250(%rip), %xmm0 ; 250(%rip) contains the constant 1.2f
ucomiss 12(%rsp), %xmm0 ; 12(%rsp) contains the variable b
Notice that the floating-point constant 1.2f
did not get embedded into
the comparison instruction
ucomiss. The
constant had to be fetched (with
movss) into the
register xmm0
before the comparsion could take place. To summarize,
a floating-point comparison takes twice as many instructions as an integer comparsion, increasing the executable code size;
a floating-point comparison involves an extra fetch instruction (movss
),
potentially causing a
cache miss.
Caveats. As we’ll see in the next section, using integer thresholds will add overhead costs at prediction time. You should ensure that the benefits of integer comparisons outweights the overhead costs.
When quantize
option is enabled, Treelite will collect all thresholds
occuring in the tree ensemble model. For each feature, one list will be
generated that lists the thresholds in ascending order:
/* example of how per-feature threshold list may look like */
Feature 0: [1.5, 6.5, 12.5]
Feature 3: [0.15, 0.35, 1.5]
Feature 6: [7, 9, 10, 135]
Using these lists, we may convert any data point into integer indices via simple look-ups. For feature 0 in the example above, values will be mapped to integer indices as follows:
Let x be the value of feature 0.
Assign -1 if x < 1.5
Assign 0 if x == 1.5
Assign 1 if 1.5 < x < 6.5
Assign 2 if x == 6.5
Assign 3 if 6.5 < x < 12.5
Assign 4 if x == 12.5
Assign 5 if x > 12.5
Let’s look at a specific example of how a floating-point vector gets translated into a vector of integer indices:
feature id 0 1 2 3 4 5 6
[7, missing, missing, 0.2, missing, missing, 20 ]
=> [3, missing, missing, 1, missing, missing, 5 ]
Since the prediction subroutine still needs to accept floating-point features,
the features will be internally converted before actual prediction. If the
prediction subroutine looked like below without quantize
option,
float predict_margin(const Entry* data) {
/* ... Run through the trees to compute the leaf output score ... */
return score;
}
it will now have an extra step of mapping the incoming data vector into integers:
float predict_margin(const Entry* data) {
/* ... Quantize feature values in data into integer indices ... */
/* ... Run through the trees to compute the leaf output score ... */
return score;
}
After all the hard work you did to train your tree ensemble model, you now have to deploy the model. Deployment refers to distributing your model to other machines and devices so as to make predictions on them. To facilitate the coming discussions, let us define a few terms.
Host machine : the machine running Treelite.
Target machine : the machine on which predictions will be made. The host machine may or may not be identical to the target machine. In cases where it’s infeasible to install Treelite on the target machine, the host and target machines will be necessarily distinct.
Shared library : a blob of executable subroutines that can be imported by other native applications. Shared libraries will often have file extensions .dll, .so, or .dylib. Going back to the particular context of tree deployment, Treelite will produce a shared library containing the prediction subroutine (compiled to native machine code).
Runtime package : a tiny fraction of the full Treelite package, consisting of a few helper functions that lets you easily load shared libraries and make predictions. The runtime is good to have, but on systems lacking Python we can do without it.
In this document, we will document two options for deployment. We will present the programming interface each deployment option presents, as well as its dependencies and requirements.
Contents
If feasible, this option is probably the most convenient. On the target machine, install the Treelite runtime by running pip:
python3 -m pip install treelite_runtime --user
Once the Treelite runtime is installed, it suffices to follow instructions in First tutorial.
With this option, neither Python nor a C++ compiler is required. You should be able to adopt this option using any basic installation of UNIX-like operating systems.
The target machine shall meet the following conditions:
A C compiler is available.
The C compiler supports the following features of the
C99 standard: inline functions;
declaration of loop variables inside for
loop; the expf
function in
<math.h>
; the <stdint.h>
header.
GNU Make or Microsoft NMake is installed.
An archive utility exists that can open a .zip archive.
1. On the host machine, install Treelite and import your tree ensemble model.
You should end up with the model object of type Model
.
### Run this block on the **host** machine
import treelite
model = treelite.Model.load('your_model.model', 'xgboost')
# You may also use `from_xgboost` method or the builder class
2. Export your model as a source package by calling the method
export_srcpkg()
of the Model
object. The source package will contain C code representation of the prediction
subroutine.
### Continued from the previous code block
# Operating system of the target machine
platform = 'unix'
# C compiler to use to compile prediction code on the target machine
toolchain = 'gcc'
# Save the source package as a zip archive named mymodel.zip
# Later, we'll use this package to produce the library mymodel.so.
model.export_srcpkg(platform=platform, toolchain=toolchain,
pkgpath='./mymodel.zip', libname='mymodel.so',
verbose=True)
Note
On the value of toolchain
Treelite supports only three toolchain configurations (‘msvc’, ‘gcc’, ‘clang’)
for which it generates Makefiles. If you are using a compiler other than
these three, you will have to write your own Makefile. For now, just set
toolchain='gcc'
and move on.
After calling export_srcpkg()
, you should be able to
find the zip archive named mymodel.zip
inside the current working directory.
john.doe@host-machine:/home/john.doe/$ ls .
mymodel.zip your_model.model
The content of mymodel.zip
consists of the header and source files, as well
as the Makefile:
john.doe@host-machine:/home/john.doe/$ unzip -l mymodel.zip
Archive: mymodel.zip
Length Date Time Name
--------- ---------- ----- ----
0 11-01-2017 23:11 mymodel/
167 11-01-2017 23:11 mymodel/Makefile
4831036 11-01-2017 23:11 mymodel/mymodel.c
311 11-01-2017 23:11 mymodel/mymodel.h
109 11-01-2017 23:11 mymodel/recipe.json
--------- -------
4831623 5 files
3. Now you are ready to deploy the model to the target machine. Copy to the
target machine the archive mymodel.zip
(source package).
john.doe@host-machine:/home/john.doe/$ sftp john.doe@target-machine
Connected to target-machine.
sftp> put mymodel.zip
Uploading mymodel.zip to /home/john.doe/mymodel.zip
mymodel.zip 100% 410KB 618.2KB/s 00:00
sftp> quit
4. It is time to move to the target machine. On the target machine, extract
the archive mymodel.zip
:
john.doe@host-machine:/home/john.doe/$ ssh john.doe@target-machine
Last login: Tue Oct 31 00:43:36 2017 from host-machine
john.doe@target-machine:/home/john.doe/$ unzip mymodel.zip
Archive: mymodel.zip
creating: mymodel/
inflating: mymodel/Makefile
inflating: mymodel/mymodel.c
inflating: mymodel/mymodel.h
inflating: mymodel/recipe.json
5. Build the source package (using GNU Make or NMake).
john.doe@target-machine:/home/john.doe/$ cd mymodel
john.doe@target-machine:/home/john.doe/mymodel/$ make
gcc -c -O3 -o mymodel.o mymodel.c -fPIC -std=c99 -flto -fopenmp
gcc -shared -O3 -o mymodel.so mymodel.o -std=c99 -flto -fopenmp
john.doe@target-machine:/home/john.doe/mymodel/$ ls
Makefile mymodel.c mymodel.so
mymodel.h mymodel.o recipe.json
Note
Parallel compilation with GNU Make
If you used parallel_comp
option to split the model into multiple source
files, you can take advantage of parallel compilation. Simply replace make
with make -jN
, where N
is replaced with the number of workers to
launch. Setting N
too high may result into memory shortage.
Note
Using other compilers
If you are using a compiler other than gcc, clang, or Microsoft Visual C++,
you will need to compose your own Makefile. Open the Makefile
and
make necessary changes.
The prediction library provides the function predict
with the
following signature:
float predict(union Entry* data, int pred_margin);
Here, the argument data
must be an array of length M
, where M
is
the number of features used in the tree ensemble. The data
array stores
all the feature values of a single row. To indicate presence or absence of
a feature value, we use the union type Entry
, which defined as
union Entry {
int missing;
float fvalue;
};
For missing values, we set the missing
field to -1. For non-missing ones, we
set the fvalue
field to the feature value. The total number of features
is given by the function
size_t get_num_feature(void);
Let’s look at an example. We’d start by initializing the array inst
, a dense
aray to hold feature values of a single data row:
/* number of features */
const size_t num_feature = get_num_feature();
/* inst: dense vector storing feature values */
union Entry* inst = malloc(sizeof(union Entry) * num_feature);
/* clear inst with all missing values */
for (i = 0; i < num_feature; ++i) {
inst[i].missing = -1;
}
Before calling the function predict
, the array inst
needs to be
initialized with missing and present feature values. The following peudocode
illustrates the idea:
For each data row rid:
inst[i].missing == -1 for every i, assuming all features lack values
For each feature i for which the data row in fact has a feature value:
Set inst[i].fvalue = [feature value], to indicate presence
Call predict(inst, 0) and get prediction for the data row rid
For each feature i for which the row has a feature value:
Set inst[i].missing = -1, to prepare for next row (rid + 1)
The task is not too difficult as long as the input data is given as a particular form of sparse matrix: the Compressed Sparse Row format. The sparse matrix consists of three arrays:
val
stores nonzero entries in
row-major order.
col_ind
stores column indices of the entries in val
. The expression
col_ind[i]
indicates the column index of the i
th entry val[i]
.
row_ptr
stores the locations in val
that start and end data rows. The
i
th data row is given by the array slice val[row_ptr[i]:row_ptr[i+1]]
.
/* nrow : number of data rows */
for (rid = 0; rid < nrow; ++rid) {
ibegin = row_ptr[rid];
iend = row_ptr[rid + 1];
/* Fill nonzeros */
for (i = ibegin; i < iend; ++i) {
inst[col_ind[i]].fvalue = val[i];
}
out_pred[rid] = predict(inst, 0);
/* Drop nonzeros */
for (i = ibegin; i < iend; ++i) {
inst[col_ind[i]].missing = -1;
}
}
It only remains to create three arrays val
, col_ind
, and row_ptr
.
You may want to use a third-pary library here to read from
a SVMLight format. For now, we’ll punt the issue of loading the input data
and write it out as constants in the program:
#include <stdio.h>
#include <stdlib.h>
#include "mymodel.h"
int main(void) {
/* 5x13 "sparse" matrix, in CSR format
[[ 0. , 0. , 0.68, 0.99, 0. , 0.11, 0. , 0.82, 0. ,
0. , 0. , 0. , 0. ],
[ 0. , 0. , 0.99, 0. , 0. , 0. , 0. , 0. , 0. ,
0.61, 0. , 0. , 0. ],
[ 0.02, 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ],
[ 0. , 0. , 0.36, 0. , 0.82, 0. , 0. , 0.57, 0. ,
0. , 0. , 0. , 0.75],
[ 0.47, 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0.45, 0. ]]
*/
const float val[] = {0.68, 0.99, 0.11, 0.82, 0.99, 0.61, 0.02, 0.36, 0.82,
0.57, 0.75, 0.47, 0.45};
const size_t col_ind[] = {2, 3, 5, 7, 2, 9, 0, 2, 4, 7, 12, 0, 11};
const size_t row_ptr[] = {0, 4, 6, 7, 11, 13};
const size_t nrow = 5;
const size_t ncol = 13;
/* number of features */
const size_t num_feature = get_num_feature();
/* inst: dense vector storing feature values */
union Entry* inst = malloc(sizeof(union Entry) * num_feature);
float* out_pred = malloc(sizeof(float) * nrow);
size_t rid, ibegin, iend, i;
/* clear inst with all missing */
for (i = 0; i < num_feature; ++i) {
inst[i].missing = -1;
}
for (rid = 0; rid < nrow; ++rid) {
ibegin = row_ptr[rid];
iend = row_ptr[rid + 1];
/* Fill nonzeros */
for (i = ibegin; i < iend; ++i) {
inst[col_ind[i]].fvalue = val[i];
}
out_pred[rid] = predict(inst, 0);
/* Drop nonzeros */
for (i = ibegin; i < iend; ++i) {
inst[col_ind[i]].missing = -1;
}
printf("pred[%zu] = %f\n", rid, out_pred[rid]);
}
free(inst);
free(out_pred);
return 0;
}
Save the program as a .c file and put it in the same directory mymodel/
. To
link the program against the prediction library mymodel.so
, simply run
gcc -o myprog myprog.c mymodel.so -I. -std=c99
As long as the program myprog
is in the same directory of the prediction
library mymodel.so
, we’ll be good to go.
A sample output:
pred[0] = 44.880001
pred[1] = 44.880001
pred[2] = 44.880001
pred[3] = 42.670002
pred[4] = 44.880001
Treelite4J is the Java runtime for Treelite. This tutorial will show how to use Treelite4J to deploy decision tree models to Java applications.
Locate the compiled model (dll/so/dylib) in the local filesystem. We load the compiled model by creating a Predictor object:
import ml.dmlc.treelite4j.Predictor;
Predictor predictor = new Predictor("path/to/compiled_model.so", -1, true, true);
The second argument is set to -1, to utilize all CPU cores available. See here for the meaning of third and fourth arguments.
Once the compiled model is loaded, we can query it:
// Get the input dimension, i.e. the number of feature values in the input vector
int num_feature = predictor.GetNumFeature();
// Get the number of classes.
// This number is 1 for tasks other than multi-class classification.
// For multi-class classification task, the number is equal to the number of classes.
int num_class = predictor.GetNumClass();
For predicting with a single input, we create an array of Entry objects, set their values, and invoke the prediction function.
import ml.dmlc.treelite4j.Entry;
// Create an array of feature values for the input
int num_feature = predictor.GetNumFeature();
Entry[] inst = new Entry[num_feature];
// Initialize all feature values as missing
for (int i = 0; i < num_feature; ++i) {
inst[i] = new Entry();
inst[i].setMissing();
}
// Set feature values that are not missing
// In this example, we set feature 1, 3, and 7
inst[1].setFValue(-0.5);
inst[3].setFValue(3.2);
inst[7].setFValue(-1.7);
// Now run prediction
// (Put false in the second argument to get probability outputs)
float[] result = predictor.predict(inst, false);
// The result is either class probabilities (for multi-class classification)
// or a single number (for all other tasks, such as regression)
For predicting with a batch of inputs, we create a list of DataPoint objects. Each DataPoint object consists of feature values and corresponding feature indices.
Let us look at an example. Consider the following 4-by-6 data matrix
where the dot (.) indicates the missing value. The matrix consists of 4 data points (instances), each with 6 feature values. Since not all feature values are present, we need to store feature indices as well as feature values:
import ml.dmlc.treelite4j.DataPoint;
// Create a list consisting of 4 data points
List<DataPoint> dmat = new ArrayList<DataPoint>() {
{
// feature indices feature values
add(new DataPoint(new int[]{0, 1}, new float[]{10f, 20f}));
add(new DataPoint(new int[]{1, 3}, new float[]{30f, 40f}));
add(new DataPoint(new int[]{2, 3, 4}, new float[]{50f, 60f, 70f}));
add(new DataPoint(new int[]{5}, new float[]{80f}));
}
};
Once the list is created, we then convert it into a SparseBatch object. We use SparseBatch rather than DenseBatch because significant portion of the data matrix consists of missing values.
import ml.dmlc.treelite4j.BatchBuilder;
// Convert data point list into SparseBatch object
SparseBatch batch = BatchBuilder.CreateSparseBatch(dmat);
Now invoke the batch prediction function using the SparseBatch object:
// verbose=true, pred_margin=false
float[][] result = predictor.predict(batch, true, false);
The returned array is a two-dimensional array where the array result[i]
represents the prediction for the i
-th data point. For most applications, each result[i]
has length 1. Multi-class classification task is specical, in that for that task result[i]
contains class probabilities, so the array is as long as the number of target classes.
For your convenience, we also provide a convenience method to load a data text file in the LIBSVM format:
List<DataPoint> dmat = BatchBuilder.LoadDatasetFromLibSVM("path/to/my.data.libsvm");
SparseBatch batch = BatchBuilder.CreateSparseBatch(dmat);
Since the scope of Treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.
Using XGBoost or LightGBM for training? Read this document instead.
Contents
The ModelBuilder
class is a tool used to specify decision
tree ensembles programmatically. Each tree ensemble is represented as follows:
Each Tree
object is a dictionary of
nodes indexed by unique integer keys.
A node is either a leaf node or a test node. A test node specifies its left and right children by their integer keys in the tree dictionary.
Each ModelBuilder
object is a list of
Tree
objects.
Consider the following tree ensemble, consisting of two regression trees:
Note
Provision for missing data: default directions
Decision trees in treelite accomodate missing data by indicating the default direction for every test node. In the diagram above, the default direction is indicated by label “Missing.” For instance, the root node of the first tree shown above will send to the left all data points that lack values for feature 0.
For now, let’s assume that we’ve somehow found optimal choices of default directions at training time. For detailed instructions for actually deciding default directions, see Section 3.4 of the XGBoost paper.
Let us construct this ensemble using the model builder. First step is to assign unique integer key to each node. In the following diagram, integer keys are indicated in red. Note that integer keys need to be unique only within the same tree.
Next, we create a model builder object by calling the constructor for
ModelBuilder
, with an num_feature
argument indicating
the total number of features used in the ensemble:
import treelite
builder = treelite.ModelBuilder(num_feature=3)
We also create a tree object; it will represent the first tree in the ensemble.
# to represent the first tree
tree = treelite.ModelBuilder.Tree()
The first tree has five nodes, each of which is to be inserted into the tree one at a time. The syntax for node insertion is as follows:
tree[0] # insert a new node with key 0
Once a node has been inserted, we can refer to it by writing
tree[0] # refer to existing node #0
The meaning of the expression tree[0]
thus depends on whether the node #0
exists in the tree or not.
We may combine node insertion with a function call to specify its content.
For instance, node #0 is a test node, so we call
set_numerical_test_node()
:
# Node #0: feature 0 < 5.0 ? (default direction left)
tree[0].set_numerical_test_node(feature_id=0,
opname='<',
threshold=5.0,
default_left=True,
left_child_key=1,
right_child_key=2)
On the other hand, node #2 is a leaf node, so call
set_leaf_node()
instead:
# Node #2: leaf with output +0.6
tree[2].set_leaf_node(0.6)
Let’s go ahead and specify the other three nodes:
# Node #1: feature 2 < -3.0 ? (default direction right)
tree[1].set_numerical_test_node(feature_id=2,
opname='<',
threshold=-3.0,
default_left=False,
left_child_key=3,
right_child_key=4)
# Node #3: leaf with output -0.4
tree[3].set_leaf_node(-0.4)
# Node #4: leaf with output +1.2
tree[4].set_leaf_node(1.2)
We must indicate which node is the root:
# Set node #0 as root
tree[0].set_root()
We are now done with the first tree. We insert it with the model builder
by calling append()
. (Recall that the model
builder is really a list of tree objects, hence the method name append
.)
# Insert the first tree into the ensemble
builder.append(tree)
The second tree is constructed analogously:
tree2 = treelite.ModelBuilder.Tree()
# Node #0: feature 1 < 2.5 ? (default direction right)
tree2[0].set_numerical_test_node(feature_id=1,
opname='<',
threshold=2.5,
default_left=False,
left_child_key=1,
right_child_key=2)
# Set node #0 as root
tree2[0].set_root()
# Node #1: leaf with output +1.6
tree2[1].set_leaf_node(1.6)
# Node #2: feature 2 < -1.2 ? (default direction left)
tree2[2].set_numerical_test_node(feature_id=2,
opname='<',
threshold=-1.2,
default_left=True,
left_child_key=3,
right_child_key=4)
# Node #3: leaf with output +0.1
tree2[3].set_leaf_node(0.1)
# Node #4: leaf with output -0.3
tree2[4].set_leaf_node(-0.3)
# Insert the second tree into the ensemble
builder.append(tree2)
We are now done building the member trees. The last step is to call
commit()
to finalize the ensemble into
a Model
object:
# Finalize and obtain Model object
model = builder.commit()
Note
Difference between ModelBuilder
and
Model
objects
Why does Treelite require one last step of “committing”? All
Model
objects are immutable; once constructed,
they cannot be modified at all. So you won’t be able to add a tree or a node
to an existing Model
object, for instance. On the other
hand, ModelBuilder
objects are mutable, so that you
can iteratively build trees.
To ensure we got all details right, we can examine the resulting C program.
model.compile(dirpath='./test')
with open('./test/test.c', 'r') as f:
for line in f.readlines():
print(line, end='')
which produces the output
/* Other functions omitted for space consideration */
float predict_margin(union Entry* data) {
float sum = 0.0f;
if (!(data[0].missing != -1) || data[0].fvalue < 5) {
if ( (data[2].missing != -1) && data[2].fvalue < -3) {
sum += (float)-0.4;
} else {
sum += (float)1.2;
}
} else {
sum += (float)0.6;
}
if ( (data[1].missing != -1) && data[1].fvalue < 2.5) {
sum += (float)1.6;
} else {
if (!(data[2].missing != -1) || data[2].fvalue < -1.2) {
sum += (float)0.1;
} else {
sum += (float)-0.3;
}
}
return sum + (0);
}
The toy example has been helpful as an illustration, but it is impractical to manually specify nodes for real-world ensemble models. The following section will show us how to automate the tree building process. We will look at scikit-learn in particular.
Scikit-learn (scikit-learn/scikit-learn) is a Python machine learning package known for its versatility and ease of use. It supports a wide variety of models and algorithms.
Treelite will be able to work with any decision tree ensemble models produced by scikit-learn. In particular, it will be able to work with
Note
Why scikit-learn? How about other packages?
We had to pick a specific example for programmatic tree construction, so we chose scikit-learn. If you’re using another package, don’t lose heart. As you read through the rest of section, notice how specific pieces of information about the tree ensemble model are being extracted. As long as your choice of package exposes equivalent information, you’ll be able to adapt the example to your needs.
Note
In a hurry? Try the sklearn module
The rest of this document explains in detail how to import scikit-learn
models using the builder class. If you prefer to skip all the gory details,
simply import the module treelite.sklearn
.
import treelite.sklearn
model = treelite.sklearn.import_model(clf)
Note
Adaboost ensembles not yet supported
Treelite currently does not support weighting of member trees, so you won’t be able to use Adaboost ensembles.
Let’s start with the Boston house prices dataset, a regression problem. (Classification problems are somewhat trickier, so we’ll save them for later.)
We’ll be using RandomForestRegressor
, a random
forest for regression. A random forest is an ensemble of decision trees
that are independently trained on random samples from the training data. See
this page for
more details. For now, just remember to specify random_forest=True
in the
ModelBuilder
constructor.
import sklearn.datasets
import sklearn.ensemble
# Load the Boston housing dataset
X, y = sklearn.datasets.load_boston(return_X_y=True)
# Train a random forest regressor with 10 trees
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)
We shall programmatically construct Tree
objects from internal attributes of the scikit-learn model. We only need
to define a few helper functions.
For the rest of sections, we’ll be diving into lots of details that are specific to scikit-learn. Many details have been adopted from this reference page.
The function process_model() takes in a scikit-learn ensemble object and
returns the completed Model
object:
@classmethod
def process_model(cls, sklearn_model):
"""Process a RandomForestRegressor to convert it into a Treelite model"""
# Initialize Treelite model builder
# Set average_tree_output=True for random forests
builder = treelite.ModelBuilder(
num_feature=sklearn_model.n_features_, average_tree_output=True,
threshold_type='float64', leaf_output_type='float64')
# Iterate over individual trees
for i in range(sklearn_model.n_estimators):
# Process the i-th tree and add to the builder
# process_tree() to be defined later
builder.append(cls.process_tree(sklearn_model.estimators_[i].tree_,
sklearn_model))
return builder.commit()
The usage of this function is as follows:
from treelite.sklearn import SKLRFRegressorConverter
model = SKLRFRegressorConverter.process_model(clf)
We won’t have space here to discuss all internals of scikit-learn objects, but a few details should be noted:
The attribute n_features_
stores the number of features used anywhere
in the tree ensemble.
The attribute n_estimators
stores the number of member trees.
The attribute estimators_
is an array of handles that store the individual
member trees. To access the object for the i
-th tree, write
estimators_[i].tree_
. This object will be passed to the function
process_tree()
.
The function process_tree() takes in a single scikit-learn tree object
and returns an object of type Tree
:
@classmethod
def process_tree(cls, sklearn_tree, sklearn_model):
"""Process a scikit-learn Tree object"""
treelite_tree = treelite.ModelBuilder.Tree(
threshold_type='float64', leaf_output_type='float64')
# Iterate over each node: node ID ranges from 0 to [node_count]-1
for node_id in range(sklearn_tree.node_count):
cls.process_node(treelite_tree, sklearn_tree, node_id, sklearn_model)
# Node #0 is always root for scikit-learn decision trees
treelite_tree[0].set_root()
return treelite_tree
Explanations:
The attribute node_count
stores the number of nodes in the decision tree.
Each node in the tree has a unique ID ranging from 0 to [node_count]-1
.
The function process_node() determines whether each node is a leaf node
or a test node. It does so by looking at the attribute children_left
:
If the left child of the node is set to -1, that node is thought to be
a leaf node.
@classmethod
def process_node(cls, treelite_tree, sklearn_tree, node_id, sklearn_model):
"""Process a tree node in a scikit-learn Tree object. Decide whether the node is
a leaf node or a test node."""
if sklearn_tree.children_left[node_id] == -1: # leaf node
cls.process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model)
else: # test node
cls.process_test_node(treelite_tree, sklearn_tree, node_id, sklearn_model)
The function process_test_node() extracts the content of a test node
and passes it to the Tree
object that is
being constructed.
@classmethod
def process_test_node(cls, treelite_tree, sklearn_tree, node_id, sklearn_model):
# pylint: disable=W0613
"""Process a test node with a given node ID. We shall assume that all tree ensembles in
scikit-learn use only numerical splits."""
treelite_tree[node_id].set_numerical_test_node(
feature_id=sklearn_tree.feature[node_id],
opname='<=',
threshold=sklearn_tree.threshold[node_id],
threshold_type='float64',
default_left=True,
left_child_key=sklearn_tree.children_left[node_id],
right_child_key=sklearn_tree.children_right[node_id],)
Explanations:
The attribute feature
is the array containing feature indices used
in test nodes.
The attribute threshold
is the array containing threshold values used
in test nodes.
All tests are in the form of [feature value] <= [threshold]
.
The attributes children_left
and children_right
together store
children’s IDs for test nodes.
Note
Scikit-learn and missing data
Scikit-learn handles missing data differently than XGBoost and Treelite.
Before training an ensemble model, you’ll have to impute
missing values. For this reason, test nodes in scikit-learn tree models will
contain no “default direction.” We will assign default_left=True
arbitrarily for test nodes to keep Treelite happy.
The function process_leaf_node() defines a leaf node:
@classmethod
def process_leaf_node(cls, treelite_tree, sklearn_tree, node_id, sklearn_model):
# pylint: disable=W0613
"""Process a test node with a given node ID"""
# The `value` attribute stores the output for every leaf node.
leaf_value = sklearn_tree.value[node_id].squeeze()
# Initialize the leaf node with given node ID
treelite_tree[node_id].set_leaf_node(leaf_value, leaf_value_type='float64')
Let’s test it out:
from treelite.sklearn import SKLRFRegressorConverter
model = SKLRFRegressorConverter.process_model(clf)
model.export_lib(libpath='./libtest.dylib', toolchain='gcc', verbose=True)
import treelite_runtime
predictor = treelite_runtime.Predictor(libpath='./libtest.dylib')
predictor.predict(treelite_runtime.DMatrix(X))
Gradient boosting is an algorithm where decision trees are trained one at a
time, ensuring that latter trees complement former trees. See this page
for more details. Treelite makes distinction between random forests and
gradient boosted trees by the value of random_forest
flag in the
ModelBuilder
constructor.
Note
Set init='zero'
to ensure compatibility
To make sure that the gradient boosted model is compatible with Treelite,
make sure to set init='zero'
in the
GradientBoostingRegressor
constructor. This
ensures that the compiled prediction subroutine will produce the correct
prediction output. Gradient boosting models trained without specifying
init='zero'
in the constructor are NOT supported by Treelite!
# Gradient boosting regressor
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingRegressor(n_estimators=10,
init='zero')
clf.fit(X, y)
We will recycle most of the helper code we wrote earlier. Only two functions will need to be modified:
@classmethod
def process_model(cls, sklearn_model):
"""Process a GradientBoostingRegressor to convert it into a Treelite model"""
# Check for init='zero'
if sklearn_model.init != 'zero':
raise treelite.TreeliteError("Gradient boosted trees must be trained with "
"the option init='zero'")
# Initialize Treelite model builder
# Set average_tree_output=False for gradient boosted trees
builder = treelite.ModelBuilder(
num_feature=sklearn_model.n_features_, average_tree_output=False,
threshold_type='float64', leaf_output_type='float64')
for i in range(sklearn_model.n_estimators):
# Process i-th tree and add to the builder
builder.append(cls.process_tree(sklearn_model.estimators_[i][0].tree_,
sklearn_model))
return builder.commit()
@classmethod
def process_leaf_node(cls, treelite_tree, sklearn_tree, node_id, sklearn_model):
"""Process a test node with a given node ID"""
leaf_value = sklearn_tree.value[node_id].squeeze()
# Need to shrink each leaf output by the learning rate
leaf_value *= sklearn_model.learning_rate
# Initialize the leaf node with given node ID
treelite_tree[node_id].set_leaf_node(leaf_value, leaf_value_type='float64')
Some details specific to GradientBoostingRegressor
:
To indicate the use of gradient boosting (as opposed to random forests), we
set random_forest=False
in the ModelBuilder
constructor.
Each tree object is now accessed with the expression
estimators_[i][0].tree_
, as estimators_[i]
returns an array consisting
of a single tree (i
-th tree).
Each leaf output in gradient boosted trees are “unscaled”: it needs to be scaled by the learning rate.
Let’s test it:
from treelite.sklearn import SKLGBMRegressorConverter
# Convert to Treelite model
model = SKLGBMRegressorConverter.process_model(clf)
# Generate shared library
model.export_lib(libpath='./libtest2.dylib', toolchain='gcc', verbose=True)
# Make prediction with predictor
predictor = treelite_runtime.Predictor(libpath='./libtest2.dylib')
predictor.predict(treelite_runtime.DMatrix(X))
For binary classification, let’s use the digits dataset. We will take 0’s and 1’s from the dataset and treat 0’s as the negative class and 1’s as the positive.
# load a binary classification problem
# Set n_class=2 to produce two classes
digits = sklearn.datasets.load_digits(n_class=2)
X, y = digits['data'], digits['target']
# Should print [0 1]
print(np.unique(y))
# Train a random forest classifier
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)
Random forest classifiers in scikit-learn store frequency counts for the positive and negative class. For instance, a leaf node may output a set of counts
[ 100, 200 ]
which indicates the following:
300 data points in the training set “belong” to this leaf node, in the sense that they all satisfy the precise sequence of conditions leading to that particular leaf node. The picture below shows that each leaf node represents a unique sequence of conditions:
100 of them are labeled negative; and
the remaining 200 are labeled positive.
Again, most of the helper functions may be re-used; only two functions need to be rewritten. Explanation will follow after the code:
@classmethod
def process_model(cls, sklearn_model):
"""Process a RandomForestClassifier (binary classifier) to convert it into a
Treelite model"""
builder = treelite.ModelBuilder(
num_feature=sklearn_model.n_features_, average_tree_output=True,
threshold_type='float64', leaf_output_type='float64')
for i in range(sklearn_model.n_estimators):
# Process i-th tree and add to the builder
builder.append(cls.process_tree(sklearn_model.estimators_[i].tree_,
sklearn_model))
return builder.commit()
@classmethod
def process_leaf_node(cls, treelite_tree, sklearn_tree, node_id, sklearn_model):
# pylint: disable=W0613
"""Process a test node with a given node ID"""
# Get counts for each label (+/-) at this leaf node
leaf_count = sklearn_tree.value[node_id].squeeze()
# Compute the fraction of positive data points at this leaf node
fraction_positive = float(leaf_count[1]) / leaf_count.sum()
# The fraction above is now the leaf output
treelite_tree[node_id].set_leaf_node(fraction_positive, leaf_value_type='float64')
As noted earlier, we access the frequency counts at each leaf node, reading the
value
attribute of each tree. Then we compute the fraction of positive
data points with respect to all training data points belonging to the leaf.
This fraction then becomes the leaf output. This way, leaf nodes now produce
single numbers rather than frequency count arrays.
Why did we have to compute a fraction? For binary classification, Treelite expects each tree to produce a single number output. At prediction time, the outputs from the member trees will get averaged to produce the final prediction, which is also a single number. By setting the positive fraction as the leaf output, we ensure that the final prediction is a proper probability value. For instance, if an ensemble consisting of 5 trees produces the following set of outputs
Tree 0 0.1
Tree 1 0.7
Tree 2 0.4
Tree 3 0.3
Tree 4 0.7
then the final prediction will be 0.44, which we interpret as 44% probability for the positive class.
Let’s use the digits dataset again, this time with 4 classes (i.e. 0’s, 1’s, 2’s, and 3’s).
# Load a multi-class classification problem
# Set n_class=4 to produce four classes
digits = sklearn.datasets.load_digits(n_class=4)
X, y = digits['data'], digits['target']
# Should print [0 1 2 3]
print(np.unique(y))
# Train a random forest classifier
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)
Random forest classifiers in scikit-learn store frequency counts (see the explanation in the previous section). For instance, a leaf node may output a set of counts
[ 100, 400, 300, 200 ]
which shows that the total of 1000 training data points belong to this leaf node and that 100, 400, 300, and 200 of them are labeled class 0, 1, 2, and 3, respectively.
We will have to re-write the process_leaf_node() function to accomodate multiple classes.
@classmethod
def process_model(cls, sklearn_model):
"""Process a RandomForestClassifier (multi-class classifier) to convert it into a
Treelite model"""
# Must specify num_class and pred_transform
builder = treelite.ModelBuilder(
num_feature=sklearn_model.n_features_, num_class=sklearn_model.n_classes_,
average_tree_output=True, pred_transform='identity_multiclass',
threshold_type='float64', leaf_output_type='float64')
for i in range(sklearn_model.n_estimators):
# Process i-th tree and add to the builder
builder.append(cls.process_tree(sklearn_model.estimators_[i].tree_,
sklearn_model))
return builder.commit()
@classmethod
def process_leaf_node(cls, treelite_tree, sklearn_tree, node_id, sklearn_model):
# pylint: disable=W0613
"""Process a test node with a given node ID"""
# Get counts for each label class at this leaf node
leaf_count = sklearn_tree.value[node_id].squeeze()
# Compute the probability distribution over label classes
prob_distribution = leaf_count / leaf_count.sum()
# The leaf output is the probability distribution
treelite_tree[node_id].set_leaf_node(prob_distribution, leaf_value_type='float64')
The process_leaf_node()
function is quite similar to what we had for the
binary classification case. Only difference is that, instead of computing the
fraction of the positive class, we compute the probability distribution for
all possible classes. Each leaf node thus will store the probability
distribution of possible class outcomes.
The process_model()
function is also similar to what we had before. The
crucial difference is the existence of parameters num_class
and
pred_transform
. The num_class
parameter is used only for
multi-class classification: it should store the number of classes (in this
example, 4). The pred_transform
parameter should be set to
'identity_multiclass'
, to indicate
that the prediction should be made simply by averaging the probability
distribution produced by each leaf node. (Leaf outputs are averaged rather
than summed because we set random_forest=True
.) For instance, if an ensemble
consisting of 3 trees produces the following set of outputs
Tree 0 [ 0.5, 0.5, 0.0, 0.0 ]
Tree 1 [ 0.1, 0.5, 0.3, 0.1 ]
Tree 2 [ 0.2, 0.5, 0.2, 0.1 ]
then the final prediction will be the average
[ 0.26666667, 0.5, 0.16666667, 0.06666667 ]
, which indicates 26.7%
probability for the first class, 50.0% for the second, 16.7% for the third,
and 6.7% for the fourth.
We use the digits dataset. We will take 0’s and 1’s from the dataset and treat 0’s as the negative class and 1’s as the positive.
# Load a binary classification problem
# Set n_class=2 to produce two classes
digits = sklearn.datasets.load_digits(n_class=2)
X, y = digits['data'], digits['target']
# Should print [0 1]
print(np.unique(y))
# Train a gradient boosting classifier
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingClassifier(n_estimators=10,
init='zero')
clf.fit(X, y)
Note
Set init='zero'
to ensure compatibility
To make sure that the gradient boosted model is compatible with Treelite,
make sure to set init='zero'
in the
GradientBoostingClassifier
constructor. This
ensures that the compiled prediction subroutine will produce the correct
prediction output. Gradient boosting models trained without specifying
init='zero'
in the constructor are NOT supported by Treelite!
Here are the functions process_model()
and process_leaf_node()
for this
scenario:
@classmethod
def process_model(cls, sklearn_model):
"""Process a GradientBoostingClassifier (binary classifier) to convert it into a
Treelite model"""
# Check for init='zero'
if sklearn_model.init != 'zero':
raise treelite.TreeliteError("Gradient boosted trees must be trained with "
"the option init='zero'")
# Initialize Treelite model builder
# Set average_tree_output=False for gradient boosted trees
# Set pred_transform='sigmoid' to obtain probability predictions
builder = treelite.ModelBuilder(
num_feature=sklearn_model.n_features_, average_tree_output=False,
pred_transform='sigmoid', threshold_type='float64', leaf_output_type='float64')
for i in range(sklearn_model.n_estimators):
# Process i-th tree and add to the builder
builder.append(cls.process_tree(sklearn_model.estimators_[i][0].tree_,
sklearn_model))
return builder.commit()
@classmethod
def process_leaf_node(cls, treelite_tree, sklearn_tree, node_id, sklearn_model):
"""Process a test node with a given node ID"""
leaf_value = sklearn_tree.value[node_id].squeeze()
# Need to shrink each leaf output by the learning rate
leaf_value *= sklearn_model.learning_rate
# Initialize the leaf node with given node ID
treelite_tree[node_id].set_leaf_node(leaf_value, leaf_value_type='float64')
Some details specific to GradientBoostingClassifier
:
To indicate the use of gradient boosting (as opposed to random forests), we
set random_forest=False
in the ModelBuilder
constructor.
Each tree object is now accessed with the expression
estimators_[i][0].tree_
, as estimators_[i]
returns an array consisting
of a single tree (i
-th tree).
Each leaf output in gradient boosted trees are “unscaled”: it needs to be scaled by the learning rate.
In addition, we specify the parameter pred_transform='sigmoid'
so that
the final prediction yields the probability for the positive class. For example,
suppose that an ensemble consisting of 4 trees produces the following set of
outputs:
Tree 0 +0.5
Tree 1 -2.3
Tree 2 +1.5
Tree 3 -1.5
Unlike the random forest example earlier, we do not assume that each leaf output is between 0 and 1; it can be any real number, negative or positive. These numbers are referred to as margin scores, to distinguish them from probabilities.
To obtain the probability for the positive class, we first sum the margin scores (outputs) from the member trees.
Tree 0 +0.5
Tree 1 -2.3
Tree 2 +1.5
Tree 3 -1.5
--------------
Total -1.8
Then we apply the sigmoid function:
The resulting value is the final prediction. You may interpret this value as a probability. For the particular example, the sigmoid value of -1.8 is 0.14185106, which we interpret as 14.2% probability for the positive class.
Let’s use the digits dataset again, this time with 4 classes (i.e. 0’s, 1’s, 2’s, and 3’s).
# Load a multi-class classification problem
# Set n_class=4 to produce four classes
digits = sklearn.datasets.load_digits(n_class=4)
X, y = digits['data'], digits['target']
# Should print [0 1 2 3]
print(np.unique(y))
# Train a gradient boosting classifier
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingClassifier(n_estimators=10,
init='zero')
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)
Note
Set init='zero'
to ensure compatibility
To make sure that the gradient boosted model is compatible with Treelite,
make sure to set init='zero'
in the
GradientBoostingClassifier
constructor. This
ensures that the compiled prediction subroutine will produce the correct
prediction output. Gradient boosting models trained without specifying
init='zero'
in the constructor are NOT supported by Treelite!
Here are the functions process_model()
and process_leaf_node()
for this
scenario:
@classmethod
def process_model(cls, sklearn_model):
"""Process a GradientBoostingClassifier (multi-class classifier) to convert it into a
Treelite model"""
# Check for init='zero'
if sklearn_model.init != 'zero':
raise treelite.TreeliteError("Gradient boosted trees must be trained with "
"the option init='zero'")
# Initialize Treelite model builder
# Set average_tree_output=False for gradient boosted trees
# Set num_class for multi-class classification
# Set pred_transform='softmax' to obtain probability predictions
builder = treelite.ModelBuilder(
num_feature=sklearn_model.n_features_, num_class=sklearn_model.n_classes_,
average_tree_output=False, pred_transform='softmax',
threshold_type='float64', leaf_output_type='float64')
# Process [number of iterations] * [number of classes] trees
for i in range(sklearn_model.n_estimators):
for k in range(sklearn_model.n_classes_):
builder.append(cls.process_tree(sklearn_model.estimators_[i][k].tree_,
sklearn_model))
return builder.commit()
@classmethod
def process_leaf_node(cls, treelite_tree, sklearn_tree, node_id, sklearn_model):
"""Process a test node with a given node ID"""
leaf_value = sklearn_tree.value[node_id].squeeze()
# Need to shrink each leaf output by the learning rate
leaf_value *= sklearn_model.learning_rate
# Initialize the leaf node with given node ID
treelite_tree[node_id].set_leaf_node(leaf_value, leaf_value_type='float64')
The process_leaf_node()
function is identical to one in the previous
section: as before, each leaf node produces a single real-number output.
On the other hand, the process_model()
function needs some explanation.
First of all, the attribute estimators_
of the scikit-learn model object
now stores output groups, which are simply groups of decision trees.
The expression estimators_[i]
thus refers to the i
th output group.
Each output group contains as many trees as there are label classes. For the
digits example with 4 label classes, we’d have 4 trees for each output group:
estimators_[i][0]
, estimators_[i][1]
, estimators_[i][2]
, and
estimators_[i][3]
. Since there are as many output groups as the number of
iterations used for training, the total number of member trees is
[number of iterations] * [number of classes]
. We have to call append()
once for each member tree; hence the use of nested loop.
We also set pred_transform='softmax'
, which indicates the way margin
outputs should be transformed to produce probability predictions. Let us look
at a concrete example: suppose we train an ensemble model with 3 rounds of
gradient boosting. It would produce a total of 12 decision trees (3 rounds *
4 classes). Suppose also that, given a single test data point, the model
produces the following set of margins:
Output group 0:
Tree 0 produces +0.5
Tree 1 produces +1.5
Tree 2 produces -2.3
Tree 3 produces -1.5
Output group 1:
Tree 4 produces +0.1
Tree 5 produces +0.7
Tree 6 produces +1.5
Tree 7 produces -0.9
Output group 2:
Tree 8 produces -0.1
Tree 9 produces +0.3
Tree 10 produces -0.7
Tree 11 produces +0.2
How do we compute probabilities for each of the 4 classes? First, we compute the sum of the margin scores for each output group:
Output group 0:
Tree 0 produces +0.5
Tree 1 produces +1.5
Tree 2 produces -2.3
Tree 3 produces -1.5
----------------------
SUBTOTAL -1.8
Output group 1:
Tree 4 produces +0.1
Tree 5 produces +0.7
Tree 6 produces +1.5
Tree 7 produces -0.9
----------------------
SUBTOTAL +1.4
Output group 2:
Tree 8 produces -0.1
Tree 9 produces +0.3
Tree 10 produces -0.7
Tree 11 produces +0.2
----------------------
SUBTOTAL -0.3
The vector [-1.8, +1.4, -0.3]
consisting of the subtotals quantifies the
relative likelihood of the label classes. Since the second element (1.4) is
the largest, the second class must be the most likely outcome for the particular
data point. This vector is not yet a probability distribution, since its
elements do not sum to 1.
The softmax function transforms any real-valued vector into a probability distribution as follows:
Apply the exponential function (exp
) to every element in the vector.
This step ensures that every element is positive.
Divide every element by the sum over the vector. This step is also known as normalizing the vector. After thie step, the elements of the vector will add up to 1.
Let’s walk through the steps with the vector [-1.8, +1.4, -0.3]
. Applying
the exponential function is simple with Python:
x = np.exp([-1.8, +1.4, -0.3])
print(x)
which yields
[ 0.16529889 4.05519997 0.74081822]
Note that every element is now positive. Then we normalize the vector by writing
x = x / x.sum()
print(x)
which gives a proper probability distribution:
[ 0.03331754 0.8173636 0.14931886]
We can now interpret the result as giving 3.3% probability for the first class, 81.7% probability for the second, and 14.9% probability for the third.
API of Treelite Python package.
Treelite: a model compiler for decision tree ensembles
treelite.
Model
(handle=None)¶Decision tree ensemble model
handle (ctypes.c_void_p
, optional) – Initial value of model handle
compile
(dirpath, params=None, compiler='ast_native', verbose=False)¶Generate prediction code from a tree ensemble model. The code will be C99
compliant. One header file (.h) will be generated, along with one or more
source files (.c). Use create_shared()
method to package
prediction code as a dynamic shared library (.so/.dll/.dylib).
Example
The following populates the directory ./model
with source and header
files:
model.compile(dirpath='./my/model', params={}, verbose=True)
If parallel compilation is enabled (parameter parallel_comp
), the files
are in the form of ./my/model/header.h
, ./my/model/main.c
,
./my/model/tu0.c
, ./my/model/tu1.c
and so forth, depending on
the value of parallel_comp
. Otherwise, there will be exactly two files:
./model/header.h
, ./my/model/main.c
deserialize
(filename)¶Deserialize (recover) the model from a checkpoint file in the disk. It is expected that
the file was generated by a call to the serialize()
method.
Note
Use exactly matching versions of Treelite when exchanging checkpoints
We provide ZERO backward compatibility guarantee. You will not be able to recover the model from a checkpoint that was generated by a previous version of Treelite. Both the producer and the consumer of the checkpoint must have the identical major and minor versions of Treelite.
export_lib
(toolchain, libpath, params=None, compiler='ast_native', verbose=False, nthread=None, options=None)¶Convenience function: Generate prediction code and immediately turn it into a dynamic shared library. A temporary directory will be created to hold the source files.
toolchain (str
) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’.
You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)
libpath (str
) – location to save the generated dynamic shared library
params (dict
, optional) – parameters to be passed to the compiler. See
this page for the list of compiler
parameters.
compiler (str
, optional) – name of compiler to use in C code generation
verbose (bool
, optional) – whether to produce extra messages
nthread (int
, optional) – number of threads to use in creating the shared library.
Defaults to the number of cores in the system.
options (list
of str
, optional) – Additional options to pass to toolchain
Example
The one-line command
model.export_lib(toolchain='msvc', libpath='./mymodel.dll',
params={}, verbose=True)
is equivalent to the following sequence of commands:
model.compile(dirpath='/temporary/directory', params={}, verbose=True)
treelite.create_shared(toolchain='msvc', dirpath='/temporary/directory',
verbose=True)
# move the library out of the temporary directory
shutil.move('/temporary/directory/mymodel.dll', './mymodel.dll')
export_srcpkg
(platform, toolchain, pkgpath, libname, params=None, compiler='ast_native', verbose=False, options=None)¶Convenience function: Generate prediction code and create a zipped source package for deployment. The resulting zip file will also contain a Makefile.
platform (str
) – name of the operating system on which the headers and sources shall be
compiled. Must be one of the following: ‘windows’ (Microsoft Windows),
‘osx’ (Mac OS X), ‘unix’ (Linux and other UNIX-like systems)
toolchain (str
) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, ‘gcc’, and ‘cmake’.
You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)
pkgpath (str
) – location to save the zipped source package
libname (str
) – name of model shared library to be built
params (dict
, optional) – parameters to be passed to the compiler. See
this page for the list of compiler
parameters.
compiler (str
, optional) – name of compiler to use in C code generation
verbose (bool
, optional) – whether to produce extra messages
nthread (int
, optional) – number of threads to use in creating the shared library.
Defaults to the number of cores in the system.
options (list
of str
, optional) – Additional options to pass to toolchain
Example
The one-line command
model.export_srcpkg(platform='unix', toolchain='gcc',
pkgpath='./mymodel_pkg.zip', libname='mymodel.so',
params={}, verbose=True)
is equivalent to the following sequence of commands:
model.compile(dirpath='/temporary/directory/mymodel',
params={}, verbose=True)
generate_makefile(dirpath='/temporary/directory/mymodel',
platform='unix', toolchain='gcc')
# zip the directory containing C code and Makefile
shutil.make_archive(base_name=pkgpath, format='zip',
root_dir='/temporary/directory',
base_dir='mymodel/')
from_xgboost
(booster)¶Load a tree ensemble model from an XGBoost Booster object
booster (object of type xgboost.Booster
) – Python handle to XGBoost model
model – loaded model
Model
object
Example
bst = xgboost.train(params, dtrain, 10, [(dtrain, 'train')])
xgb_model = Model.from_xgboost(bst)
from_xgboost_json
(json_str)¶Load a tree ensemble model from a string containing XGBoost JSON
json_str (a string specifying an XGBoost model in the XGBoost JSON) – format
model – loaded model
Model
object
Example
bst = xgboost.train(params, dtrain, 10, [(dtrain, 'train')])
bst.save_model('model.json')
with open('model.json') as file_:
json_str = file_.read()
xgb_model = Model.from_xgboost_json(json_str)
load
(filename, model_format)¶Load a tree ensemble model from a file
model – loaded model
Model
object
Example
xgb_model = Model.load('xgboost_model.model', 'xgboost')
num_class
¶Number of classes of the model (1 if the model is not a multi-class classifier
num_feature
¶Number of features used in the model
num_tree
¶Number of decision trees in the model
serialize
(filename)¶Serialize (persist) the model to a checkpoint file in the disk, using a fast binary
representation. To recover the model from the checkpoint, use deserialize()
method.
Note
Use exactly matching versions of Treelite when exchanging checkpoints
We provide ZERO backward compatibility guarantee. You will not be able to recover the model from a checkpoint that was generated by a previous version of Treelite. Both the producer and the consumer of the checkpoint must have the identical major and minor versions of Treelite.
filename (str
) – Path to checkpoint
set_tree_limit
(tree_limit)¶Set first n trees to be kept, the remaining ones will be dropped
treelite.
ModelBuilder
(num_feature, num_class=1, average_tree_output=False, threshold_type='float32', leaf_output_type='float32', **kwargs)¶Builder class for tree ensemble model: provides tools to iteratively build an ensemble of decision trees
num_feature (int
) – number of features used in model being built. We assume that all
feature indices are between 0
and (num_feature - 1
)
num_class (int
, optional) – number of output groups; >1
indicates multiclass classification
average_tree_output (bool
, optional) – whether the model is a random forest; True
indicates a random forest
and False
indicates gradient boosted trees
**kwargs – model parameters, to be used to specify the resulting model. Refer to this page for the full list of model parameters.
Node
¶Handle to a node in a tree
set_categorical_test_node
(feature_id, left_categories, default_left, left_child_key, right_child_key)¶Set the node as a test node with categorical split. A list defines all
categories that would be classified as the left side. Categories are
integers ranging from 0
to n-1
, where n
is the number of
categories in that particular feature.
feature_id (int
) – feature index
left_categories (list
of int
) – list of categories belonging to the left child.
default_left (bool
) – default direction for missing values
(True
for left; False
for right)
left_child_key (int
) – unique integer key to identify the left child node
right_child_key (int
) – unique integer key to identify the right child node
set_leaf_node
(leaf_value, leaf_value_type='float32')¶Set the node as a leaf node
set_numerical_test_node
(feature_id, opname, threshold, default_left, left_child_key, right_child_key, threshold_type='float32')¶Set the node as a test node with numerical split. The test is in the form
[feature value] OP [threshold]
. Depending on the result of the test,
either left or right child would be taken.
feature_id (int
) – feature index
opname (str
) – binary operator to use in the test
threshold (float
) – threshold value
default_left (bool
) – default direction for missing values
(True
for left; False
for right)
left_child_key (int
) – unique integer key to identify the left child node
right_child_key (int
) – unique integer key to identify the right child node
threshold_type (str) – data type for threshold value (e.g. ‘float32’)
set_root
()¶Set the node as the root
Tree
(threshold_type='float32', leaf_output_type='float32')¶Handle to a decision tree in a tree ensemble Builder
Value
(init_value, dtype)¶Value whose type may be specified at runtime
dtype (str) – Initial value of model handle
append
(tree)¶Add a tree at the end of the ensemble
tree (Tree
object) – tree to be added
Example
builder = ModelBuilder(num_feature=4227)
tree = ... # build tree somehow
builder.append(tree) # add tree at the end of the ensemble
commit
()¶Finalize the ensemble model
model – finished model
Model
object
Example
builder = ModelBuilder(num_feature=4227)
for i in range(100):
tree = ... # build tree somehow
builder.append(tree) # add one tree at a time
model = builder.commit() # now get a Model object
model.compile(dirpath='test') # compile model into C code
insert
(index, tree)¶Insert a tree at specified location in the ensemble
Example
builder = ModelBuilder(num_feature=4227)
tree = ... # build tree somehow
builder.insert(0, tree) # insert tree at index 0
treelite.
Annotator
¶Branch annotator class: annotate branches in a given model using frequency patterns in the training data
annotate_branch
(model, dmat, nthread=None, verbose=False)¶Annotate branches in a given model using frequency patterns in the training data. Each node gets the count of the instances that belong to it. Any prior annotation information stored in the annotator will be replaced with the new annotation returned by this method.
model (object of type Model
) – decision tree ensemble model
dmat (object of type DMatrix
) – data matrix representing the training data
nthread (int
, optional) – number of threads to use while annotating. If missing, use all physical
cores in the system.
verbose (bool
, optional) – whether to produce extra messages
Create shared library.
toolchain (str
) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’.
You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)
dirpath (str
) – directory containing the header and source files previously generated
by Model.compile()
. The directory must contain recipe.json
which specifies build dependencies.
nthread (int
, optional) – number of threads to use in creating the shared library.
Defaults to the number of cores in the system.
verbose (bool
, optional) – whether to produce extra messages
options (list
of str
, optional) – Additional options to pass to toolchain
libpath – absolute path of created shared library
Example
The following command uses Visual C++ toolchain to generate
./my/model/model.dll
:
model.compile(dirpath='./my/model', params={}, verbose=True)
create_shared(toolchain='msvc', dirpath='./my/model', verbose=True)
Later, the shared library can be referred to by its directory name:
predictor = Predictor(libpath='./my/model', verbose=True)
# looks for ./my/model/model.dll
Alternatively, one may specify the library down to its file name:
predictor = Predictor(libpath='./my/model/model.dll', verbose=True)
treelite.
generate_makefile
(dirpath, platform, toolchain, options=None)¶Generate a Makefile for a given directory of headers and sources. The resulting Makefile will be stored in the directory. This function is useful for deploying a model on a different machine.
dirpath (str
) – directory containing the header and source files previously generated
by Model.compile()
. The directory must contain recipe.json
which specifies build dependencies.
platform (str
) – name of the operating system on which the headers and sources shall be
compiled. Must be one of the following: ‘windows’ (Microsoft Windows),
‘osx’ (Mac OS X), ‘unix’ (Linux and other UNIX-like systems)
toolchain (str
) – which toolchain to use. You may choose one of ‘msvc’, ‘clang’, and ‘gcc’.
You may also specify a specific variation of clang or gcc (e.g. ‘gcc-7’)
options (list
of str
, optional) – Additional options to pass to toolchain
treelite.
generate_cmakelists
(dirpath, options=None)¶Generate a CMakeLists.txt for a given directory of headers and sources. The resulting CMakeLists.txt will be stored in the directory. This function is useful for deploying a model on a different machine.
dirpath (str
) – directory containing the header and source files previously generated
by Model.compile()
. The directory must contain recipe.json
which specifies build dependencies.
options (list
of str
, optional) – Additional options to pass to toolchain
treelite.
TreeliteError
¶Error thrown by Treelite
Converter to ingest scikit-learn models into Treelite
treelite.sklearn.
import_model_with_model_builder
(sklearn_model)¶Load a tree ensemble model from a scikit-learn model object using the model builder API.
Note
Use import_model
for production use
This function exists to demonstrate the use of the model builder API and is slow with
large models. For production, please use import_model()
which is significantly faster.
sklearn_model (object of type RandomForestRegressor
/ RandomForestClassifier
/ ExtraTreesRegressor
/ ExtraTreesClassifier
/ GradientBoostingRegressor
/ GradientBoostingClassifier
) – Python handle to scikit-learn model
model – loaded model
Model
object
Example
import sklearn.datasets
import sklearn.ensemble
X, y = sklearn.datasets.load_boston(return_X_y=True)
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)
import treelite.sklearn
model = treelite.sklearn.import_model_with_model_builder(clf)
treelite.sklearn.
import_model
(sklearn_model)¶Load a tree ensemble model from a scikit-learn model object
sklearn_model (object of type RandomForestRegressor
/ RandomForestClassifier
/ ExtraTreesRegressor
/ ExtraTreesClassifier
/ GradientBoostingRegressor
/ GradientBoostingClassifier
) – Python handle to scikit-learn model
model – loaded model
Model
object
Example
import sklearn.datasets
import sklearn.ensemble
X, y = sklearn.datasets.load_boston(return_X_y=True)
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)
import treelite.sklearn
model = treelite.sklearn.import_model(clf)
General Tree Inference Library (GTIL)
treelite.gtil.
predict
(model: treelite.frontend.Model, data: numpy.ndarray, pred_margin: bool = False)¶Predict with a Treelite model using General Tree Inference Library (GTIL). GTIL is intended to be a reference implementation. GTIL is also useful in situations where using a C compiler is not feasible.
Note
GTIL is currently experimental
GTIL is currently in its early stage of development and may have bugs and performance issues. Please report any issues found on GitHub.
model (Model
object) – Treelite model object
data (numpy.ndarray
array) – 2D NumPy array, with which to run prediction
pred_margin (bool
, optional) – Whether to produce raw margin scores
prediction – Prediction
numpy.ndarray
array
Runtime API of Treelite Python package.
treelite_runtime.
Predictor
(libpath, nthread=None, verbose=False)¶Predictor class: loader for compiled shared libraries
Note
Treelite uses a custom thread pool which pins threads to CPU cores by default.
To disable thread pinning, set the environment variable
TREELITE_BIND_THREADS
to 0
. Disabling thread pinning is recommended when
using Treelite in multi-threaded applications.
global_bias
¶Query global bias of the model
leaf_output_type
¶Query threshold type of the model
num_class
¶Query number of output groups of the model
num_feature
¶Query number of features used in the model
pred_transform
¶Query pred transform of the model
predict
(dmat, verbose=False, pred_margin=False)¶Perform batch prediction with a 2D sparse data matrix. Worker threads will internally divide up work for batch prediction. Note that this function may be called by only one thread at a time.
sigmoid_alpha
¶Query sigmoid alpha of the model
threshold_type
¶Query threshold type of the model
treelite_runtime.
DMatrix
(data, data_format=None, dtype=None, missing=None, feature_names=None, feature_types=None, verbose=False, nthread=None)¶Data matrix used in Treelite.
data (str
/ numpy.ndarray
/ scipy.sparse.csr_matrix
/ pandas.DataFrame
) – Data source. When data is str
type, it indicates
that data should be read from a file.
data_format (str
, optional) – Format of input data file. Applicable only when data is read from a
file. If missing, the svmlight (.libsvm) format is assumed.
dtype (str
, optional) – If specified, the data will be casted into the corresponding data type.
missing (float
, optional) – Value in the data that represents a missing entry. If set to None
,
numpy.nan
will be used.
verbose (bool
, optional) – Whether to print extra messages during construction
feature_names (list
, optional) – Human-readable names for features
feature_types (list
, optional) – Types for features
nthread (int
, optional) – Number of threads
treelite_runtime.
TreeliteRuntimeError
¶Error thrown by Treelite runtime
Treelite exposes a set of C functions to enable interfacing with a variety of languages. This page will be most useful for:
those writing a new language binding (glue code).
those wanting to incorporate functions of Treelite into their own native libraries.
We recommend the Python API for everyday uses.
Note
Use of C and C++ in Treelite
Core logic of Treelite are written in C++ to take advantage of higher abstractions. We provide C only interface here, as many more programming languages bind with C than with C++. See this page for more details.
Contents
Use the following functions to load and manipulate data from a variety of sources.
TreeliteDMatrixCreateFromFile
(const char *path, const char *format, const char *data_type, int nthread, int verbose, DMatrixHandle *out)¶create a sparse DMatrix from a file
0 for success, -1 for failure
path
: file path
format
: file format
nthread
: number of threads to use
verbose
: whether to produce extra messages
out
: the created DMatrix
TreeliteDMatrixCreateFromCSR
(const void *data, const char *data_type, const uint32_t *col_ind, const size_t *row_ptr, size_t num_row, size_t num_col, DMatrixHandle *out)¶create DMatrix from a (in-memory) CSR matrix
0 for success, -1 for failure
data
: feature values
data_type
: Type of data elements
col_ind
: feature indices
row_ptr
: pointer to row headers
num_row
: number of rows
num_col
: number of columns
out
: the created DMatrix
TreeliteDMatrixCreateFromMat
(const void *data, const char *data_type, size_t num_row, size_t num_col, const void *missing_value, DMatrixHandle *out)¶create DMatrix from a (in-memory) dense matrix
0 for success, -1 for failure
data
: feature values
data_type
: Type of data elements
num_row
: number of rows
num_col
: number of columns
missing_value
: value to represent missing value
out
: the created DMatrix
TreeliteDMatrixGetDimension
(DMatrixHandle handle, size_t *out_num_row, size_t *out_num_col, size_t *out_nelem)¶get dimensions of a DMatrix
0 for success, -1 for failure
handle
: handle to DMatrix
out_num_row
: used to set number of rows
out_num_col
: used to set number of columns
out_nelem
: used to set number of nonzero entries
TreeliteDMatrixFree
(DMatrixHandle handle)¶delete DMatrix from memory
0 for success, -1 for failure
handle
: handle to DMatrix
Use the following functions to annotate branches in decision trees.
TreeliteAnnotateBranch
(ModelHandle model, DMatrixHandle dmat, int nthread, int verbose, AnnotationHandle *out)¶annotate branches in a given model using frequency patterns in the training data.
0 for success, -1 for failure
model
: model to annotate
dmat
: training data matrix
nthread
: number of threads to use
verbose
: whether to produce extra messages
out
: used to save handle for the created annotation
TreeliteAnnotationSave
(AnnotationHandle handle, const char *path)¶save branch annotation to a JSON file
0 for success, -1 for failure
handle
: annotation to save
path
: path to JSON file
TreeliteAnnotationFree
(AnnotationHandle handle)¶delete branch annotation from memory
0 for success, -1 for failure
handle
: annotation to remove
Use the following functions to produce optimize prediction subroutine (in C) from a given decision tree ensemble.
TreeliteCompilerCreate
(const char *name, CompilerHandle *out)¶create a compiler with a given name
0 for success, -1 for failure
name
: name of compiler
out
: created compiler
TreeliteCompilerSetParam
(CompilerHandle handle, const char *name, const char *value)¶set a parameter for a compiler
0 for success, -1 for failure
handle
: compiler
name
: name of parameter
value
: value of parameter
TreeliteCompilerGenerateCode
(CompilerHandle compiler, ModelHandle model, int verbose, const char *dirpath)¶generate prediction code from a tree ensemble model. The code will be C99 compliant. One header file (.h) will be generated, along with one or more source files (.c).
Usage example:
TreeliteCompilerGenerateCode(compiler, model, 1, "./my/model");
// files to generate: ./my/model/header.h, ./my/model/main.c
// if parallel compilation is enabled:
// ./my/model/header.h, ./my/model/main.c, ./my/model/tu0.c,
// ./my/model/tu1.c, and so forth
0 for success, -1 for failure
compiler
: handle for compiler
model
: handle for tree ensemble model
verbose
: whether to produce extra messages
dirpath
: directory to store header and source files
TreeliteCompilerFree
(CompilerHandle handle)¶delete compiler from memory
0 for success, -1 for failure
handle
: compiler to remove
Use the following functions to load decision tree ensemble models from a file. Treelite supports multiple model file formats.
TreeliteLoadLightGBMModel
(const char *filename, ModelHandle *out)¶load a model file generated by LightGBM (Microsoft/LightGBM). The model file must contain a decision tree ensemble.
0 for success, -1 for failure
filename
: name of model file
out
: loaded model
TreeliteLoadXGBoostModel
(const char *filename, ModelHandle *out)¶load a model file generated by XGBoost (dmlc/xgboost). The model file must contain a decision tree ensemble.
0 for success, -1 for failure
filename
: name of model file
out
: loaded model
TreeliteLoadXGBoostJSON
(const char *filename, ModelHandle *out)¶load a json model file generated by XGBoost (dmlc/xgboost). The model file must contain a decision tree ensemble.
0 for success, -1 for failure
filename
: name of model file
out
: loaded model
TreeliteLoadXGBoostJSONString
(const char *json_str, size_t length, ModelHandle *out)¶load a model stored as JSON stringby XGBoost (dmlc/xgboost). The model json must contain a decision tree ensemble.
0 for success, -1 for failure
json_str
: the string containing the JSON model specification
length
: the length of the JSON string
out
: loaded model
TreeliteLoadXGBoostModelFromMemoryBuffer
(const void *buf, size_t len, ModelHandle *out)¶load an XGBoost model from a memory buffer.
0 for success, -1 for failure
buf
: memory buffer
len
: size of memory buffer
out
: loaded model
TreeliteLoadSKLearnRandomForestRegressor
(int n_estimators, int n_features, const int64_t *node_count, const int64_t **children_left, const int64_t **children_right, const int64_t **feature, const double **threshold, const double **value, const int64_t **n_node_samples, const double **impurity, ModelHandle *out)¶Load a scikit-learn random forest regressor model from a collection of arrays. Refer to https://scikit-learn.org/stable/auto_examples/tree/plot_unveil_tree_structure.html to learn the mearning of the arrays in detail. Note that this function can also be used to load an ensemble of extremely randomized trees (sklearn.ensemble.ExtraTreesRegressor).
0 for success, -1 for failure
n_estimators
: number of trees in the random forest
n_features
: number of features in the training data
node_count
: node_count[i] stores the number of nodes in the i-th tree
children_left
: children_left[i][k] stores the ID of the left child node of node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
children_right
: children_right[i][k] stores the ID of the right child node of node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
feature
: feature[i][k] stores the ID of the feature used in the binary tree split at node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
threshold
: threshold[i][k] stores the threshold used in the binary tree split at node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
value
: value[i][k] stores the leaf output of node k of the i-th tree. This is only defined if node k is a leaf node.
n_node_samples
: n_node_samples[i][k] stores the number of data samples associated with node k of the i-th tree.
impurity
: impurity[i][k] stores the impurity measure (gini, entropy etc) associated with node k of the i-th tree.
out
: pointer to store the loaded model
TreeliteLoadSKLearnRandomForestClassifier
(int n_estimators, int n_features, int n_classes, const int64_t *node_count, const int64_t **children_left, const int64_t **children_right, const int64_t **feature, const double **threshold, const double **value, const int64_t **n_node_samples, const double **impurity, ModelHandle *out)¶Load a scikit-learn random forest classifier model from a collection of arrays. Refer to https://scikit-learn.org/stable/auto_examples/tree/plot_unveil_tree_structure.html to learn the mearning of the arrays in detail. Note that this function can also be used to load an ensemble of extremely randomized trees (sklearn.ensemble.ExtraTreesClassifier).
0 for success, -1 for failure
n_estimators
: number of trees in the random forest
n_features
: number of features in the training data
n_classes
: number of classes in the target variable
node_count
: node_count[i] stores the number of nodes in the i-th tree
children_left
: children_left[i][k] stores the ID of the left child node of node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
children_right
: children_right[i][k] stores the ID of the right child node of node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
feature
: feature[i][k] stores the ID of the feature used in the binary tree split at node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
threshold
: threshold[i][k] stores the threshold used in the binary tree split at node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
value
: value[i][k] stores the leaf output of node k of the i-th tree. This is only defined if node k is a leaf node.
n_node_samples
: n_node_samples[i][k] stores the number of data samples associated with node k of the i-th tree.
impurity
: impurity[i][k] stores the impurity measure (gini, entropy etc) associated with node k of the i-th tree.
out
: pointer to store the loaded model
TreeliteLoadSKLearnGradientBoostingRegressor
(int n_estimators, int n_features, const int64_t *node_count, const int64_t **children_left, const int64_t **children_right, const int64_t **feature, const double **threshold, const double **value, const int64_t **n_node_samples, const double **impurity, ModelHandle *out)¶Load a scikit-learn gradient boosting regressor model from a collection of arrays. Refer to https://scikit-learn.org/stable/auto_examples/tree/plot_unveil_tree_structure.html to learn the mearning of the arrays in detail.
0 for success, -1 for failure
n_estimators
: number of trees in the random forest
n_features
: number of features in the training data
node_count
: node_count[i] stores the number of nodes in the i-th tree
children_left
: children_left[i][k] stores the ID of the left child node of node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
children_right
: children_right[i][k] stores the ID of the right child node of node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
feature
: feature[i][k] stores the ID of the feature used in the binary tree split at node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
threshold
: threshold[i][k] stores the threshold used in the binary tree split at node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
value
: value[i][k] stores the leaf output of node k of the i-th tree. This is only defined if node k is a leaf node.
n_node_samples
: n_node_samples[i][k] stores the number of data samples associated with node k of the i-th tree.
impurity
: impurity[i][k] stores the impurity measure (gini, entropy etc) associated with node k of the i-th tree.
out
: pointer to store the loaded model
TreeliteLoadSKLearnGradientBoostingClassifier
(int n_estimators, int n_features, int n_classes, const int64_t *node_count, const int64_t **children_left, const int64_t **children_right, const int64_t **feature, const double **threshold, const double **value, const int64_t **n_node_samples, const double **impurity, ModelHandle *out)¶Load a scikit-learn gradient boosting classifier model from a collection of arrays. Refer to https://scikit-learn.org/stable/auto_examples/tree/plot_unveil_tree_structure.html to learn the mearning of the arrays in detail.
0 for success, -1 for failure
n_estimators
: number of trees in the random forest
n_features
: number of features in the training data
n_classes
: number of classes in the target variable
node_count
: node_count[i] stores the number of nodes in the i-th tree
children_left
: children_left[i][k] stores the ID of the left child node of node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
children_right
: children_right[i][k] stores the ID of the right child node of node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
feature
: feature[i][k] stores the ID of the feature used in the binary tree split at node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
threshold
: threshold[i][k] stores the threshold used in the binary tree split at node k of the i-th tree. This is only defined if node k is an internal (non-leaf) node.
value
: value[i][k] stores the leaf output of node k of the i-th tree. This is only defined if node k is a leaf node.
n_node_samples
: n_node_samples[i][k] stores the number of data samples associated with node k of the i-th tree.
impurity
: impurity[i][k] stores the impurity measure (gini, entropy etc) associated with node k of the i-th tree.
out
: pointer to store the loaded model
TreeliteQueryNumTree
(ModelHandle handle, size_t *out)¶Query the number of trees in the model.
0 for success, -1 for failure
handle
: model to query
out
: number of trees
TreeliteQueryNumFeature
(ModelHandle handle, size_t *out)¶Query the number of features used in the model.
0 for success, -1 for failure
handle
: model to query
out
: number of features
TreeliteQueryNumClass
(ModelHandle handle, size_t *out)¶Query the number of classes of the model. (1 if the model is binary classifier or regressor)
0 for success, -1 for failure
handle
: model to query
out
: number of output groups
TreeliteSetTreeLimit
(ModelHandle handle, size_t limit)¶keep first N trees of model, limit must smaller than number of trees.
0 for success, -1 for failure
handle
: model
limit
: number of trees to keep
TreeliteSerializeModel
(const char *filename, ModelHandle handle)¶Serialize (persist) a model object to disk.
0 for success, -1 for failure
filename
: name of the file to which to serialize the model. The file will be using a binary format that’s optimized to store the Treelite model object efficiently.
handle
: handle to the model object
TreeliteDeserializeModel
(const char *filename, ModelHandle *out)¶Deserialize (load) a model object from disk.
0 for success, -1 for failure
filename
: name of the file from which to deserialize the model. The file should be created by a call to TreeliteSerializeModel().
handle
: handle to the model object
TreeliteFreeModel
(ModelHandle handle)¶delete model from memory
0 for success, -1 for failure
handle
: model to remove
Use the following functions to incrementally build decisio n tree ensemble models.
TreeliteTreeBuilderCreateValue
(const void *init_value, const char *type, ValueHandle *out)¶Create a new Value object. Some model builder API functions accept this Value type to accommodate values of multiple types.
0 for success; -1 for failure
init_value
: pointer to the value to be stored
type
: Type of the value to be stored
out
: newly created Value object
TreeliteTreeBuilderDeleteValue
(ValueHandle handle)¶Delete a Value object from memory.
0 for success; -1 for failure
handle
: pointer to the Value object to be deleted
TreeliteCreateTreeBuilder
(const char *threshold_type, const char *leaf_output_type, TreeBuilderHandle *out)¶Create a new tree builder.
0 for success; -1 for failure
threshold_type
: Type of thresholds in numerical splits. All thresholds in a given model must have the same type.
leaf_output_type
: Type of leaf outputs. All leaf outputs in a given model must have the same type.
out
: newly created tree builder
TreeliteDeleteTreeBuilder
(TreeBuilderHandle handle)¶Delete a tree builder from memory.
0 for success; -1 for failure
handle
: tree builder to remove
TreeliteTreeBuilderCreateNode
(TreeBuilderHandle handle, int node_key)¶Create an empty node within a tree.
0 for success; -1 for failure
handle
: tree builder
node_key
: unique integer key to identify the new node
TreeliteTreeBuilderDeleteNode
(TreeBuilderHandle handle, int node_key)¶Remove a node from a tree.
0 for success; -1 for failure
handle
: tree builder
node_key
: unique integer key to identify the node to be removed
TreeliteTreeBuilderSetRootNode
(TreeBuilderHandle handle, int node_key)¶Set a node as the root of a tree.
0 for success; -1 for failure
handle
: tree builder
node_key
: unique integer key to identify the root node
TreeliteTreeBuilderSetNumericalTestNode
(TreeBuilderHandle handle, int node_key, unsigned feature_id, const char *opname, ValueHandle threshold, int default_left, int left_child_key, int right_child_key)¶Turn an empty node into a test node with numerical split. The test is in the form [feature value] OP [threshold]. Depending on the result of the test, either left or right child would be taken.
0 for success; -1 for failure
handle
: tree builder
node_key
: unique integer key to identify the node being modified; this node needs to be empty
feature_id
: id of feature
opname
: binary operator to use in the test
threshold
: threshold value
default_left
: default direction for missing values
left_child_key
: unique integer key to identify the left child node
right_child_key
: unique integer key to identify the right child node
TreeliteTreeBuilderSetCategoricalTestNode
(TreeBuilderHandle handle, int node_key, unsigned feature_id, const unsigned int *left_categories, size_t left_categories_len, int default_left, int left_child_key, int right_child_key)¶Turn an empty node into a test node with categorical split. A list defines all categories that would be classified as the left side. Categories are integers ranging from 0 to (n-1), where n is the number of categories in that particular feature. Let’s assume n <= 64.
0 for success; -1 for failure
handle
: tree builder
node_key
: unique integer key to identify the node being modified; this node needs to be empty
feature_id
: id of feature
left_categories
: list of categories belonging to the left child
left_categories_len
: length of left_cateogries
default_left
: default direction for missing values
left_child_key
: unique integer key to identify the left child node
right_child_key
: unique integer key to identify the right child node
TreeliteTreeBuilderSetLeafNode
(TreeBuilderHandle handle, int node_key, ValueHandle leaf_value)¶Turn an empty node into a leaf node.
0 for success; -1 for failure
handle
: tree builder
node_key
: unique integer key to identify the node being modified; this node needs to be empty
leaf_value
: leaf value (weight) of the leaf node
TreeliteTreeBuilderSetLeafVectorNode
(TreeBuilderHandle handle, int node_key, const ValueHandle *leaf_vector, size_t leaf_vector_len)¶Turn an empty node into a leaf vector node The leaf vector (collection of multiple leaf weights per leaf node) is useful for multi-class random forest classifier.
0 for success; -1 for failure
handle
: tree builder
node_key
: unique integer key to identify the node being modified; this node needs to be empty
leaf_vector
: leaf vector of the leaf node
leaf_vector_len
: length of leaf_vector
TreeliteCreateModelBuilder
(int num_feature, int num_class, int average_tree_output, const char *threshold_type, const char *leaf_output_type, ModelBuilderHandle *out)¶Create a new model builder.
0 for success; -1 for failure
num_feature
: number of features used in model being built. We assume that all feature indices are between 0 and (num_feature - 1).
num_class
: number of output groups. Set to 1 for binary classification and regression; >1 for multiclass classification
average_tree_output
: whether the outputs from the trees should be averaged (!=0 yes, =0 no)
threshold_type
: Type of thresholds in numerical splits. All thresholds in a given model must have the same type.
leaf_output_type
: Type of leaf outputs. All leaf outputs in a given model must have the same type.
out
: newly created model builder
TreeliteModelBuilderSetModelParam
(ModelBuilderHandle handle, const char *name, const char *value)¶Set a model parameter.
0 for success; -1 for failure
handle
: model builder
name
: name of parameter
value
: value of parameter
TreeliteDeleteModelBuilder
(ModelBuilderHandle handle)¶Delete a model builder from memory.
0 for success; -1 for failure
handle
: model builder to remove
TreeliteModelBuilderInsertTree
(ModelBuilderHandle handle, TreeBuilderHandle tree_builder, int index)¶Insert a tree at specified location.
index of the new tree within the ensemble; -1 for failure
handle
: model builder
tree_builder
: builder for the tree to be inserted. The tree must not be part of any other existing tree ensemble. Note: The tree_builder argument will become unusuable after the tree insertion. Should you want to modify the tree afterwards, use GetTree(*) method to get a fresh handle to the tree.
index
: index of the element before which to insert the tree; use -1 to insert at the end
TreeliteModelBuilderGetTree
(ModelBuilderHandle handle, int index, TreeBuilderHandle *out)¶Get a reference to a tree in the ensemble.
0 for success; -1 for failure
handle
: model builder
index
: index of the tree in the ensemble
out
: used to save reference to the tree
TreeliteModelBuilderDeleteTree
(ModelBuilderHandle handle, int index)¶Remove a tree from the ensemble.
0 for success; -1 for failure
handle
: model builder
index
: index of the tree that would be removed
TreeliteModelBuilderCommitModel
(ModelBuilderHandle handle, ModelHandle *out)¶finalize the model and produce the in-memory representation
0 for success; -1 for failure
handle
: model builder
out
: used to save handle to in-memory representation of the finished model
Use the following functions to load compiled prediction subroutines from shared libraries and to make predictions.
TreelitePredictorLoad
(const char *library_path, int num_worker_thread, PredictorHandle *out)¶load prediction code into memory. This function assumes that the prediction code has been already compiled into a dynamic shared library object (.so/.dll/.dylib).
0 for success, -1 for failure
library_path
: path to library object file containing prediction code
num_worker_thread
: number of worker threads (-1 to use max number)
out
: handle to predictor
TreelitePredictorPredictBatch
(PredictorHandle handle, DMatrixHandle batch, int verbose, int pred_margin, PredictorOutputHandle out_result, size_t *out_result_size)¶Make predictions on a batch of data rows (synchronously). This function internally divides the workload among all worker threads.
Note. This function does not allocate the result vector. Use TreeliteCreatePredictorOutputVector() convenience function to allocate the vector of the right length and type.
Note. To access the element values from the output vector, you should cast the opaque handle (PredictorOutputHandle type) to an appropriate pointer LeafOutputType*, where the type is either float, double, or uint32_t. So carry out the following steps:
Call TreelitePredictorQueryLeafOutputType() to obtain the type of the leaf output. It will return a string (“float32”, “float64”, or “uint32”) representing the type.
Depending on the type string, cast the output handle to float*, double*, or uint32_t*.
Now access the array with the casted pointer. The array’s length is given by TreelitePredictorQueryResultSize().
0 for success, -1 for failure
handle
: predictor
batch
: the data matrix containing a batch of rows
verbose
: whether to produce extra messages
pred_margin
: whether to produce raw margin scores instead of transformed probabilities
out_result
: Resulting output vector. This pointer must point to an array of length TreelitePredictorQueryResultSize() and of type TreelitePredictorQueryLeafOutputType().
out_result_size
: used to save length of the output vector, which is guaranteed to be less than or equal to TreelitePredictorQueryResultSize()
TreeliteCreatePredictorOutputVector
(PredictorHandle handle, DMatrixHandle batch, PredictorOutputHandle *out_output_vector)¶Convenience function to allocate an output vector that is able to hold the prediction result for a given data matrix. The vector’s length will be identical to TreelitePredictorQueryResultSize() and its type will be identical to TreelitePredictorQueryLeafOutputType(). To prevent memory leak, make sure to de-allocate the vector with TreeliteDeletePredictorOutputVector().
Note. To access the element values from the output vector, you should cast the opaque handle (PredictorOutputHandle type) to an appropriate pointer LeafOutputType*, where the type is either float, double, or uint32_t. So carry out the following steps:
Call TreelitePredictorQueryLeafOutputType() to obtain the type of the leaf output. It will return a string (“float32”, “float64”, or “uint32”) representing the type.
Depending on the type string, cast the output handle to float*, double*, or uint32_t*.
Now access the array with the casted pointer. The array’s length is given by TreelitePredictorQueryResultSize().
0 for success, -1 for failure
handle
: predictor
batch
: the data matrix containing a batch of rows
out_output_vector
: Handle to the newly allocated output vector.
TreeliteDeletePredictorOutputVector
(PredictorHandle handle, PredictorOutputHandle output_vector)¶De-allocate an output vector.
0 for success, -1 for failure
handle
: predictor
output_vector
: Output vector to delete from memory
TreelitePredictorQueryResultSize
(PredictorHandle handle, DMatrixHandle batch, size_t *out)¶Given a batch of data rows, query the necessary size of array to hold predictions for all data points.
0 for success, -1 for failure
handle
: predictor
batch
: the data matrix containing a batch of rows
out
: used to store the length of prediction array
TreelitePredictorQueryNumClass
(PredictorHandle handle, size_t *out)¶Get the number classes in the loaded model The number is 1 for most tasks; it is greater than 1 for multiclass classification.
0 for success, -1 for failure
handle
: predictor
out
: length of prediction array
TreelitePredictorQueryNumFeature
(PredictorHandle handle, size_t *out)¶Get the width (number of features) of each instance used to train the loaded model.
0 for success, -1 for failure
handle
: predictor
out
: number of features
TreelitePredictorQueryPredTransform
(PredictorHandle handle, const char **out)¶Get name of post prediction transformation used to train the loaded model.
0 for success, -1 for failure
handle
: predictor
out
: name of post prediction transformation
TreelitePredictorQuerySigmoidAlpha
(PredictorHandle handle, float *out)¶Get alpha value of sigmoid transformation used to train the loaded model.
0 for success, -1 for failure
handle
: predictor
out
: alpha value of sigmoid transformation
TreelitePredictorQueryGlobalBias
(PredictorHandle handle, float *out)¶Get global bias which adjusting predicted margin scores.
0 for success, -1 for failure
handle
: predictor
out
: global bias value
TreelitePredictorQueryThresholdType
(PredictorHandle handle, const char **out)¶TreelitePredictorQueryLeafOutputType
(PredictorHandle handle, const char **out)¶TreelitePredictorFree
(PredictorHandle handle)¶delete predictor from memory
0 for success, -1 for failure
handle
: predictor to remove
Treelite uses C++ classes to define its internal data structures. In order to
pass C++ objects to C functions, opaque handles are used. Opaque handles
are void*
pointers that store raw memory addresses.
ModelHandle
¶handle to a decision tree ensemble model
TreeBuilderHandle
¶handle to tree builder class
ModelBuilderHandle
¶handle to ensemble builder class
AnnotationHandle
¶handle to branch annotation data
CompilerHandle
¶handle to compiler class
ValueHandle
¶handle to a polymorphic value type, used in the model builder API
PredictorHandle
¶handle to predictor class
PredictorOutputHandle
¶handle to output from predictor
DMatrix
¶An opaque data matrix class. The actual object is stored in the C++ object handle.
Hyunsu Cho
DMatrix
(float[] data, int[] col_ind, long[] row_ptr, long num_row, long num_col)¶Create a data matrix representing a 2D sparse matrix
data – nonzero (non-missing) entries, float32 type
col_ind – corresponding column indices, should be of same length as data
row_ptr – offsets to define each instance, should be of length [num_row]+1
num_row – number of rows (data points) in the matrix
num_col – number of columns (features) in the matrix
TreeliteError – error during matrix construction
DMatrix
(double[] data, int[] col_ind, long[] row_ptr, long num_row, long num_col)¶DMatrix
(float[] data, float missing_value, long num_row, long num_col)¶Create a data matrix representing a 2D dense matrix
data – array of entries, should be of length [num_row]*[num_col]
and of float32 type
missing_value – floating-point value representing a missing value; usually set of Float.NaN
.
num_row – number of rows (data instances) in the matrix
num_col – number of columns (features) in the matrix
TreeliteError – error during matrix construction
DMatrix
(double[] data, double missing_value, long num_row, long num_col)¶Create a data matrix representing a 2D dense matrix (float64 type)
data – array of entries, should be of length [num_row]*[num_col]
and of float64 type
missing_value – floating-point value representing a missing value; usually set of Double.NaN
.
num_row – number of rows (data instances) in the matrix
num_col – number of columns (features) in the matrix
TreeliteError – error during matrix construction
dispose
()¶Destructor, to be called when the object is garbage collected
finalize
()¶getHandle
()¶Get the underlying native handle
Integer representing memory address
getNumCol
()¶Get the number of columns in the matrix
Number of columns in the matrix
getNumElements
()¶Get the number of elements in the matrix
Number of elements in the matrix
getNumRow
()¶Get the number of rows in the matrix
Number of rows in the matrix
DMatrixBuilder
¶Collection of utility functions to create data matrices
Hyunsu Cho
LoadDatasetFromLibSVM
(String filename)¶Load a LIBSVM data file and construct a list of data points
filename – path to LIBSVM file
TreeliteError – Treelite error
IOException – IO error
Created list of data points
createDenseDMatrix
(Iterator<DataPoint> dIter)¶Assemble a dense matrix from a list of data points (float32)
dIter – Iterator of data points
TreeliteError – Treelite error
IOException – IO error
Created dense batch
createSparseCSRDMatrix
(Iterator<DataPoint> dIter)¶Build a sparse (CSR layout) matrix from a list of data points (float32)
dIter – Iterator of data points
TreeliteError – Treelite error
IOException – IO error
Created sparse data matrix
DMatrixTest
¶Test cases for data matrix
Hyunsu Cho
testDenseDMatrixBasicFloat32
()¶testSparseDMatrixBasicFloat32
()¶testSparseDMatrixBuilder
()¶Data
¶Interface to specify a single data entry.
Hyunsu Cho
getFValue
()¶Obtain the floating-point value stored by the entry.
floating-point value
getQValue
()¶Obtain the integer value stored by the entry.
integer value
isMissing
()¶Test whether the entry is missing.
whether the entry is missing
setFValue
(float val)¶Assign a floating-point value to the entry.
val – value to set
setMissing
()¶Designate the entry as missing.
setQValue
(int val)¶Assign an integer value to the entry. This is useful when feature values and split thresholds are quantized into integers.
val – value to set
write
(OutputStream out)¶Serialize
out –
IOException –
Entry
extends Union implements Data¶A reference implementation for the Data
interface. This class is used to specify a single data entry. This implementation uses a C-style union so as to save memory space.
Hyunsu Cho
fvalue
¶The value of the entry, in floating-point representation
missing
¶If missing == -1
, the entry is considered missing. Otherwise, the entry is not missing; check the other fields to obtain the actual value of the entry.
qvalue
¶The value of the entry, in integer representation
getFValue
()¶Obtain the floating-point value stored by the entry.
floating-point value
getQValue
()¶Obtain the integer value stored by the entry.
integer value
isMissing
()¶Test whether the entry is missing.
whether the entry is missing
setFValue
(float val)¶Assign a floating-point value to the entry.
val – value to set
setMissing
()¶Designate the entry as missing.
setQValue
(int val)¶Assign an integer value to the entry. This is useful when feature values and split thresholds are quantized into integers.
val – value to set
InferenceEngine
¶Interface for making inference (WORK-IN-PROGRESS)
James Liu
getNumClass
()¶Get the number of classes for the compiled model. This number is 1 for tasks other than multi-class classification. For multi-class classification task, the number is equal to the number of classes.
Number of classes
getNumFeature
()¶Get the number of features used by the compiled model. Call this method to allocate array for storing data entries of a single instance.
Number of features
NativeLibLoader
¶Class to load the native lib libtreelite4j.dylib
. Normally, users do not have to worry about this class, since mvn package
automatically bundles libtreelite4j.dylib
into the JAR file. However, the method createTempFileFromResource
may be useful to some users, who would like to bundle other files into the JAR file as well.
Hyunsu Cho
createTempFileFromResource
(String path)¶Create a temp file that copies the resource from current JAR archive
The file from JAR is copied into system temp file. The temporary file is deleted after exiting. Method uses String as filename because the pathname is “abstract”, not system-dependent.
The restrictions of File.createTempFile(java.lang.String,java.lang.String)
apply to path
.
path – Path to the resources in the jar
IOException – When the temp file could not be created
IllegalArgumentException – When the file name contains invalid letters
The created temp file.
initTreeliteRuntime
()¶Initialization method to load the native treelite4j lib at startup
IOException – when treelite4j lib is not found
Predictor
implements Serializable, KryoSerializable¶Treelite Predictor
Hyunsu Cho
Predictor
(String libpath, int nthread, boolean verbose)¶Create a Predictor by loading a shared library (dll/so/dylib). The library is expected to contain the compiled code for making prediction for a particular tree ensemble model. The predictor also spawns a fixed-size pool of worker threads, who will wait for prediction tasks. (Note that the worker threads will go to sleep when no prediction task is available, to free up CPU cycles for other processes.)
libpath – Path to the shared library
nthread – Number of workers threads to spawn. Set to -1 to use default, i.e., to launch as many threads as CPU cores available on the system. You are not allowed to launch more threads than CPU cores. Setting nthread=1
indicates that the main thread should be exclusively used.
verbose – Whether to print extra diagnostic messages
TreeliteError – error during loading the shared lib
GetGlobalBias
()¶Get global bias which adjusting predicted margin scores.
Value of global bias
GetNumClass
()¶Get the number of classes for the compiled model. This number is 1 for tasks other than multi-class classification.
Number of classes
GetNumFeature
()¶Get the number of features used by the compiled model. Call this method to allocate array for storing data entries of a single instance.
Number of features
GetSigmoidAlpha
()¶Get alpha value in sigmoid transformation used to train the loaded model.
Alpha value of sigmoid transformation
dispose
()¶Destructor, to be called when the object is garbage collected
finalize
()¶predict
(DMatrix batch, boolean verbose, boolean pred_margin)¶Perform batch prediction with a 2D data matrix. Worker threads will internally divide up work for batch prediction. Note that this function will be blocked by mutex when worker_thread > 1.
batch – a data matrix of type DMatrix
verbose – whether to print extra diagnostic messages
pred_margin – whether to predict probabilities or raw margin scores
Resulting predictions, of dimension [num_row]*[num_class]
read
(Kryo kryo, Input in)¶write
(Kryo kryo, Output out)¶PredictorTest
¶Test cases for Treelite Predictor
Hyunsu Cho
PredictorTest
()¶testPredict
()¶testPredictMargin
()¶testPredictorBasic
()¶testSerialization
()¶TreeliteJNI
¶Treelite prediction runtime JNI functions
Hyunsu Cho
TreeliteDMatrixCreateFromCSRWithFloat32In
(float[] data, int[] col_ind, long[] row_ptr, long num_row, long num_col, long[] out)¶TreeliteDMatrixCreateFromCSRWithFloat64In
(double[] data, int[] col_ind, long[] row_ptr, long num_row, long num_col, long[] out)¶TreeliteDMatrixCreateFromMatWithFloat32In
(float[] data, long num_row, long num_col, float missing_value, long[] out)¶TreeliteDMatrixCreateFromMatWithFloat64In
(double[] data, long num_row, long num_col, double missing_value, long[] out)¶TreeliteDMatrixFree
(long handle)¶TreeliteDMatrixGetDimension
(long handle, long[] out_num_row, long[] out_num_col, long[] out_nelem)¶TreelitePredictorFree
(long handle)¶TreelitePredictorPredictBatchWithFloat32Out
(long handle, long batch, boolean verbose, boolean pred_margin, float[] out_result, long[] out_result_size)¶TreelitePredictorPredictBatchWithFloat64Out
(long handle, long batch, boolean verbose, boolean pred_margin, double[] out_result, long[] out_result_size)¶TreelitePredictorPredictBatchWithUInt32Out
(long handle, long batch, boolean verbose, boolean pred_margin, int[] out_result, long[] out_result_size)¶TreelitePredictorQueryGlobalBias
(long handle, float[] out)¶TreelitePredictorQueryNumClass
(long handle, long[] out)¶TreelitePredictorQueryNumFeature
(long handle, long[] out)¶TreelitePredictorQueryResultSize
(long handle, long batch, long[] out)¶TreelitePredictorQuerySigmoidAlpha
(long handle, float[] out)¶checkCall
(int ret)¶Check the return code of the JNI call.
TreeliteError – if the call failed.
Compiler parameters influence the way the prediction subroutine is generated from a tree ensemble model.
annotate_in
¶name of model annotation file. Use the class treelite.Annotator
to generate this file.
quantize
¶whether to quantize threshold points (0: no, >0: yes)
parallel_comp
¶option to enable parallel compilation; if set to nonzero, the trees will be evely distributed into [parallel_comp]
files. Set this option to improve compilation time and reduce memory consumption during compilation.
verbose
¶if >0, produce extra messages
native_lib_name
¶native lib name (without extension)
code_folding_req
¶parameter for folding rarely visited subtrees (no if/else blocks); all nodes whose data counts are lower than that of the root node of the decision tree by [code_folding_req] will be folded. To diable folding, set to +inf. If hessian sums are available, they will be used as proxies of data counts.
dump_array_as_elf
¶Only applicable when compiler is set to failsafe
. If set to a positive value, the fail-safe compiler will not emit large constant arrays to the C code. Instead, the arrays will be emitted as an ELF binary (Linux only). For large arrays, it is much faster to directly dump ELF binaries than to pass them to a C compiler.
Model parameters are used by ModelBuilder
class to
clarify certain behaviors of a tree ensemble model.
pred_transform
[TREELITE_MAX_PRED_TRANSFORM_LENGTH
] = {0}¶name of prediction transform function
This parameter specifies how to transform raw margin values into final predictions. By default, this is set to 'identity'
, which means no transformation.
For the multi-class classification task, pred_transfrom
must be one of the following values:
- identity_multiclass
do not transform. The output will be a matrix with dimensions
[number of data points] * [number of classes] that contains the margin score
for every (data point, class) pair.
- max_index
compute the most probable class for each data point and output the class
index. The output will be a vector of length [number of data points] that
contains the most likely class of each data point.
- softmax
use the softmax function to transform a multi-dimensional vector into a
proper probability distribution. The output will be a matrix with dimensions
[number of data points] * [number of classes] that contains the predicted
probability of each data point belonging to each class.
- multiclass_ova
apply the sigmoid function element-wise to the margin matrix. The output will
be a matrix with dimensions [number of data points] * [number of classes].
pred_transfrom
must be one of the following values: - identity
Do not transform. The output will be a vector of length
[number of data points] that contains the margin score for every data point.
- signed_square
Apply the function f(x) = sign(x) * (x**2) element-wise to the margin vector. The
output will be a vector of length [number of data points].
- hinge
Apply the function f(x) = (1 if x > 0 else 0) element-wise to the margin vector. The
output will be a vector of length [number of data points], filled with 0's and 1's.
- sigmoid
Apply the sigmoid function element-wise to the margin vector. The output
will be a vector of length [number of data points] that contains the
probability of each data point belonging to the positive class.
- exponential
Apply the exponential function (exp) element-wise to the margin vector. The
output will be a vector of length [number of data points].
- logarithm_one_plus_exp
Apply the function f(x) = log(1 + exp(x)) element-wise to the margin vector.
The output will be a vector of length [number of data points].
sigmoid_alpha
¶scaling parameter for sigmoid function sigmoid(x) = 1 / (1 + exp(-alpha * x))
This parameter is used only when pred_transform
is set to 'sigmoid'
. It must be strictly positive; if unspecified, it is set to 1.0.
global_bias
¶global bias of the model
Predicted margin scores of all instances will be adjusted by the global bias. If unspecified, the bias is set to zero.