Treelite is a flexible toolbox for efficient deployment of decision tree ensembles.
Star WatchYou are currently browsing the documentation of a stable version of treelite: 0.32.
Treelite compiles your tree model into optimized shared library. A Benchmark demonstrates 2-6x improvement in prediction throughput, due to more efficient use of compute resources.
Treelite accommodates a wide range of decision tree ensemble models. In particular, it handles both random forests and gradient boosted trees.
Treelite can read models produced by XGBoost, LightGBM, and scikit-learn. In cases where you are using another package to train your model, you may use the flexible builder class.
It is a great hassle to install machine learning packages (e.g. XGBoost, LightGBM, scikit-learn, etc.) on every machine your tree model will run. This is the case no longer: treelite will export your model as a stand-alone prediction library so that predictions will be made without any machine learning package installed.
Install treelite from PyPI:
pip3 install --user treelite
Import your tree ensemble model into treelite:
import treelite
model = treelite.Model.load('my_model.model', model_format='xgboost')
Deploy a source archive:
# Produce a zipped source directory, containing all model information
# Run `make` on the target machine
model.export_srcpkg(platform='unix', toolchain='gcc',
pkgpath='./mymodel.zip', libname='mymodel.so',
verbose=True)
Deploy a shared library:
# Like export_srcpkg, but generates a shared library immediately
# Use this only when the host and target machines are compatible
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True)
Make predictions on the target machine:
import treelite.runtime
predictor = treelite.runtime.Predictor('./mymodel.so', verbose=True)
batch = treelite.runtime.Batch.from_npy2d(X)
out_pred = predictor.predict(batch)
Read First tutorial for a more detailed example. See Deploying models for additional instructions on deployment.
Note
A note on API compatibility
Since treelite is in early development, its API may change substantially in the future.
The workflow involves two distinct machines: the host machine that generates prediction subroutine from a given tree model, and the target machine that runs the subroutine. The two machines exchange a single C file that contains all relevant information about the tree model. Only the host machine needs to have treelite installed; the target machine requires only a working C compiler.
You may choose one of two methods to install treelite on your system:
This is probably the most convenient method. Simply type
pip3 install --user treelite
to install the treelite package. The command will locate the binary release that is compatible with your current platform. Check the installation by running
import treelite
in an interactive Python session. This method is available for only Windows, Mac OS X, and Linux. For other operating systems, see the next section.
Installation consists of two steps:
Note
List of libraries created
There will be two libraries created: the main library, for producing optimized prediction subroutines; and the runtime library, for deploying these subroutines in the wild for actual prediction tasks.
Operating System | Main library | Runtime library |
---|---|---|
Windows | treelite.dll |
treelite_runtime.dll |
Mac OS X | libtreelite.dylib |
libtreelite_runtime.dylib |
Linux / other UNIX | libtreelite.so |
libtreelite_runtime.so |
To get started, clone treelite repo from GitHub. It is important to clone the
submodules with --recursive
option.
git clone --recursive https://github.com/dmlc/treelite.git
cd treelite
The next step is to build the shared libraries.
The Python package is located at the python
subdirectory. There are several
ways to install the package:
1. Install system-wide, which requires root permission
cd python
sudo python setup.py install
You will need Python setuptools
module for this to work. It is often part of the core Python installation.
Should it be necessary, the package can be installed using pip
:
pip install -U pip setuptools
2. Install for only current user
This is useful if you do not have the administrative rights.
cd python
python setup.py develop --user
Note
Recompiling treelite
Every time the C++ portion of treelite gets re-compiled, the Python package must be re-installed for the new library to take effect.
3. Set the environment variable PYTHONPATH to locate treelite package
Only set the environment variable PYTHONPATH
to tell Python where to find
the treelite package. This is useful for developers, as any changes made
to C++ code will be immediately visible to Python side without re-running
setup.py
.
export PYTHONPATH=path/to/treelite/python
python # enter interactive session
Note
Compiling with Protocol Buffers support
If your system has Protocol Buffers (google/protobuf) library installed, treelite will be compiled with Protocol Buffers support. That is, you will able to read tree ensemble models that had been serialized using Protocol Buffers. Protocol Buffers support is strictly optional; treelite can be compiled without it. Should you decide to use Protocol Buffers, you should specify your ensemble model according to the specification src/tree.proto.
Binary releases hosted on PyPI have been compiled with Protocol Buffers support.
On Windows, you should specify the root directory containing Protobuf
compilers and libraries by setting the environment variable
CMAKE_PREFIX_PATH
as follows:
mkdir build
cd build
:: Specify location of protobuf (Protocol Buffers)
set CMAKE_PREFIX_PATH=C:\path\to\protobuf
cmake .. -G"Visual Studio 15 2017 Win64"
This page lists tutorials about treelite.
This tutorial will demonstrate the basic workflow.
import treelite
In this tutorial, we will use a small regression example to describe the full workflow.
Let us use the Boston house prices dataset from scikit-learn
(sklearn.datasets.load_boston()
). It consists of 506 houses
with 13 distinct features:
from sklearn.datasets import load_boston
X, y = load_boston(return_X_y=True)
print('dimensions of X = {}'.format(X.shape))
print('dimensions of y = {}'.format(y.shape))
The first step is to train a tree ensemble model using XGBoost (dmlc/xgboost).
Disclaimer: Treelite does NOT depend on the XGBoost package in any way. XGBoost was used here only to provide a working example.
import xgboost
dtrain = xgboost.DMatrix(X, label=y)
params = {'max_depth':3, 'eta':1, 'silent':1, 'objective':'reg:linear',
'eval_metric':'rmse'}
bst = xgboost.train(params, dtrain, 20, [(dtrain, 'train')])
Next, we feed the trained model into treelite. If you used XGBoost to train the model, it takes only one line of code:
model = treelite.Model.from_xgboost(bst)
Note
Using other packages to train decision trees
With additional work, you can use models trained with other machine learning packages. See this page for instructions.
Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.
Contents
XGBoost (dmlc/xgboost) is a fast, scalable package for gradient boosting. Both treelite and XGBoost are hosted by the DMLC (Distributed Machine Learning Community) group.
Treelite plays well with XGBoost — if you used XGBoost to train your ensemble model, you need only one line of code to import it. Depending on where your model is located, you should do one of the following:
xgboost.Booster
object# bst = an object of type xgboost.Booster
model = Model.from_xgboost(bst)
# model had been saved to a file named my_model.model
# notice the second argument format='xgboost'
model = Model.load('my_model.model', format='xgboost')
LightGBM (Microsoft/LightGBM) is
another well known machine learning package for gradient boosting. To import
models generated by LightGBM, use the load()
method
with argument format='lightgbm'
:
# model had been saved to a file named my_model.txt
# notice the second argument format='lightgbm'
model = Model.load('my_model.txt', format='lightgbm')
Scikit-learn (scikit-learn/scikit-learn) is a Python machine learning package known for its versatility and ease of use. It supports a wide variety of models and algorithms. The following kinds of models can be imported into treelite.
sklearn.ensemble.RandomForestRegressor
sklearn.ensemble.RandomForestClassifier
sklearn.ensemble.GradientBoostingRegressor
sklearn.ensemble.GradientBoostingClassifier
To import scikit-learn models, use
treelite.gallery.sklearn.import_model()
:
# clf is the model object generated by scikit-learn
import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)
If you used other packages to train your ensemble model, you’d need to specify the model programmatically. There are two ways to do this:
Treelite offers system-level optimizations to boost prediction performance. Notice that the model information is wholly preserved; the optimizations only affect the manner at which prediction is performed.
Contents
This optimization analyzes and annotates every threshold conditions in the test nodes to improve performance.
The first step is to generate the branch annotation record for your ensemble model. Make sure to have your training data ready.
# model = your ensemble model (object of type treelite.Model)
# dmat = training data (object of type treelite.DMatrix)
# Create annotator object
annotator = treelite.Annotator()
# Annotate branches by iterating over the training data
annotator.annotate_branch(model=model, dmat=dmat, verbose=True)
# Save the branch annotation record as a JSON file
annotator.save(path='mymodel-annotation.json')
To utilize the branch annotation record, supply the compiler parameter
annotate_in
when exporting the model:
# Export a source directory
model.compile(dirpath='./mymodel', verbose=True,
params={'quantize': 1})
# Export a source directory, packaged in a zip archive
model.export_srcpkg(platform='unix', toolchain='gcc', pkgpath='./mymodel.zip',
libname='mymodel.so', verbose=True,
params={'quantize': 1})
# Export a shared library
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True,
params={'quantize': 1})
Modern CPUs heavily rely on a technique known as
branch prediction, in which
they “guess” the result of the conditional expression in each if
-else
branch ahead of time. Given a program
if ( [conditional expression] ) {
foo();
} else {
bar();
}
the CPU will pre-fetch the instructions for the function foo()
if the given
condition is likely to be true. On the other hand, if the condition is likely
to be false, the CPU will pre-fetch the instructions for the function bar()
.
It suffices to say that correctly predicting conditional branches has
great impact on performance. Each time the CPU predicts a branch correctly, it
can keep the instructions it had pre-fetched earlier. Each time the CPU fails to
predict, it must throw away the pre-fetched instructions and fetch anew another
set of instructions. If you’d like to learn more about the importance of branch
prediction, read
this excellent introductory article
from Stanford.
The prediction subroutine for a decision tree ensemble is problematic, as it is replete with conditional branches that will have to be guessed well:
/* A slice of prediction subroutine */
float predict_margin(const float* data) {
float sum = 0.0f;
if (!(data[0].missing != -1) || data[0].fvalue <= 9.5) {
if (!(data[0].missing != -1) || data[0].fvalue <= 3.5) {
if (!(data[10].missing != -1) || data[10].fvalue <= 0.74185) {
if (!(data[0].missing != -1) || data[0].fvalue <= 1.5) {
if (!(data[2].missing != -1) || data[2].fvalue <= 2.08671) {
if ( (data[4].missing != -1) && data[4].fvalue <= 2.02632) {
if (!(data[3].missing != -1) || data[3].fvalue <= 0.763339) {
sum += (float)0.00758165;
} else {
sum += (float)0.0060202;
}
} else {
if ( (data[1].missing != -1) && data[1].fvalue <= 0.0397456) {
sum += (float)0.00415399;
} else {
sum += (float)0.00821985;
}
}
/* and so forth... */
In fact, each threshold condition in the test nodes will need to be predicted. While CPUs lack adequate information to make good guesses on these conditions, we can help by providing that information.
We predict the likelihood of each condition by counting the number of data points from the training data that satisfy that condition. See the diagram below for an illustration.
If a condition is true at least 50% of the time (over the training data), the condition is labeled as “expected to be true”:
/* expected to be true */
if ( __builtin_expect( [condition], 1 ) ) {
...
} else {
...
}
On the other hand, if a condition is false at least 50% of the time, the condition is labeled as “expected to be false”:
/* expected to be false */
if ( __builtin_expect( [condition], 0 ) ) {
...
} else {
...
}
Note
On the expression __builtin_expect
The __builtin_expect
expression is a compiler intrinsic to supply the C
compiler with branch prediction information. Both
gcc
and clang
support it. Unfortunately, Microsoft Visual C++ does not. To take advantage
of branch annotation, make sure to use gcc or clang on the target machine.
This optimization replaces all thresholds in the test nodes with integers so that each threshold condition performs integer comparison instead of the usual floating-point comparison. The thresholds are said to be quantized into integer indices.
BEFORE:
if (data[3].fvalue < 1.5) { /* floating-point comparison */
...
}
AFTER:
if (data[3].qvalue < 3) { /* integer comparison */
...
}
Simply add the compiler parameter quantize=1
when exporting the model:
# Export a source directory
model.compile(dirpath='./mymodel', verbose=True,
params={'quantize': 1})
# Export a source directory, packaged in a zip archive
model.export_srcpkg(platform='unix', toolchain='gcc', pkgpath='./mymodel.zip',
libname='mymodel.so', verbose=True,
params={'quantize': 1})
# Export a shared library
model.export_lib(toolchain='gcc', libpath='./mymodel.so', verbose=True,
params={'quantize': 1})
On some platforms such as x86-64, replacing floating-point thresholds with integers helps improve performance by 1) reducing executable code size and 2) improving data locality. This is so because on these platforms, integer constants can be embedded as part of the comparison instruction, whereas floating-point constants cannot.
Let’s look at x86-64 platform. The integer comparison
a <= 4
produces one assembly instruction:
cmpl $4, 8(%rsp) ; 8(%rsp) contains the variable a
Since the integer constant 4
got embedded into the comparison instruction
cmpl, we only
had to fetch the variable a
from memory.
On the other hand, the floating-point comparison
b < 1.2f
produces two assembly instructions:
movss 250(%rip), %xmm0 ; 250(%rip) contains the constant 1.2f
ucomiss 12(%rsp), %xmm0 ; 12(%rsp) contains the variable b
Notice that the floating-point constant 1.2f
did not get embedded into
the comparison instruction
ucomiss. The
constant had to be fetched (with
movss) into the
register xmm0
before the comparsion could take place. To summarize,
movss
),
potentially causing a
cache miss.Caveats. As we’ll see in the next section, using integer thresholds will add overhead costs at prediction time. You should ensure that the benefits of integer comparisons outweights the overhead costs.
When quantize
option is enabled, treelite will collect all thresholds
occuring in the tree ensemble model. For each feature, one list will be
generated that lists the thresholds in ascending order:
/* example of how per-feature threshold list may look like */
Feature 0: [1.5, 6.5, 12.5]
Feature 3: [0.15, 0.35, 1.5]
Feature 6: [7, 9, 10, 135]
Using these lists, we may convert any data point into integer indices via simple look-ups. For feature 0 in the example above, values will be mapped to integer indices as follows:
Let x be the value of feature 0.
Assign -1 if x < 1.5
Assign 0 if x == 1.5
Assign 1 if 1.5 < x < 6.5
Assign 2 if x == 6.5
Assign 3 if 6.5 < x < 12.5
Assign 4 if x == 12.5
Assign 5 if x > 12.5
Let’s look at a specific example of how a floating-point vector gets translated into a vector of integer indices:
feature id 0 1 2 3 4 5 6
[7, missing, missing, 0.2, missing, missing, 20 ]
=> [3, missing, missing, 1, missing, missing, 5 ]
Since the prediction subroutine still needs to accept floating-point features,
the features will be internally converted before actual prediction. If the
prediction subroutine looked like below without quantize
option,
float predict_margin(const Entry* data) {
/* ... Run through the trees to compute the leaf output score ... */
return score;
}
it will now have an extra step of mapping the incoming data vector into integers:
float predict_margin(const Entry* data) {
/* ... Quantize feature values in data into integer indices ... */
/* ... Run through the trees to compute the leaf output score ... */
return score;
}
After all the hard work you did to train your tree ensemble model, you now have to deploy the model. Deployment refers to distributing your model to other machines and devices so as to make predictions on them. To facilitate the coming discussions, let us define a few terms.
In this document, we will document three options for deployment. We will present the programming interface each deployment option presents, as well as its dependencies and requirements.
Contents
If feasible, this option is probably the most convenient. On the target machine, install treelite by running pip:
pip3 install treelite --user
Once treelite is installed, it suffices to follow instructions in First tutorial.
The target machine shall meet the following conditions:
pip install
or by manual compilation, see
below)for
loop; the expf
function
in <math.h>
; the <stdint.h>
header. Recent versions of gcc and clang
should work, as well as Microsoft Visual Studio 2013 or newer.numpy
,
scipy.sparse
.In addition, if you are using operating systems other than Windows, Mac OS X, and Linux, you would need to compile treelite from the source. To do this, you’ll need git, CMake (>= 3.1), and a C++ compiler that complies with the C++11 standard.
With this option, you no longer have to install the full treelite package on the target machine. Only the runtime package and the prediction library will need to be copied to the target machine.
The target machine shall meet the following conditions:
for
loop;
the expf
function in <math.h>
; the <stdint.h>
header.numpy
,
scipy.sparse
.1. On the host machine, install treelite and import your tree ensemble model.
You should end up with the model object of type Model
.
### Run this block on the **host** machine
import treelite
model = treelite.Model.load('your_model.model', 'xgboost')
# You may also use `from_xgboost` method or the builder class
2. Export your model as a source package by calling the method
export_srcpkg()
of the Model
object. The source package will contain C code representation of the prediction
subroutine.
### Continued from the previous code block
# Operating system of the target machine
platform = 'unix'
# C compiler to use to compile prediction code on the target machine
toolchain = 'gcc'
# Save the source package as a zip archive named mymodel.zip
# Later, we'll use this package to produce the library mymodel.so.
model.export_srcpkg(platform=platform, toolchain=toolchain,
pkgpath='./mymodel.zip', libname='mymodel.so',
verbose=True)
After calling export_srcpkg()
, you should be able to
find the zip archive named mymodel.zip
inside the current working directory.
john.doe@host-machine:/home/john.doe/$ ls .
mymodel.zip your_model.model
The content of mymodel.zip
consists of the header and source files, as well
as the Makefile:
john.doe@host-machine:/home/john.doe/$ unzip -l mymodel.zip
Archive: mymodel.zip
Length Date Time Name
--------- ---------- ----- ----
0 11-01-2017 23:11 mymodel/
167 11-01-2017 23:11 mymodel/Makefile
4831036 11-01-2017 23:11 mymodel/mymodel.c
311 11-01-2017 23:11 mymodel/mymodel.h
109 11-01-2017 23:11 mymodel/recipe.json
--------- -------
4831623 5 files
3. Export the runtime package using the method
save_runtime_package()
:
### Continued from the previous code block
treelite.save_runtime_package(destdir='.')
This command produces the zip archived named treelite_runtime.zip
. This
archive contains all the necessary files for prediction task.
john.doe@host-machine:/home/john.doe/$ ls .
mymodel.zip your_model.model treelite_runtime.zip
john.doe@host-machine:/home/john.doe/$ unzip -l treelite_runtime.zip
Archive: treelite_runtime.zip
Length Date Time Name
--------- ---------- ----- ----
0 10-30-2017 12:23 runtime/
.................full output not shown here.................
709 09-26-2017 17:58 runtime/src/common/math.h
--------- -------
71353 39 files
4. Now you are ready to deploy the model to the target machine. Copy to the
target machine the two archives mymodel.zip
(source package) and
treelite_runtime.zip
(runtime package):
john.doe@host-machine:/home/john.doe/$ sftp john.doe@target-machine
Connected to target-machine.
sftp> put mymodel.zip
Uploading mymodel.zip to /home/john.doe/mymodel.zip
mymodel.zip 100% 410KB 618.2KB/s 00:00
sftp> put treelite_runtime.zip
Uploading treelite_runtime.zip to /home/john.doe/treelite_runtime.zip
treelite_runtime.zip 100% 28KB 618.0KB/s 00:00
sftp> quit
5. It is time to move to the target machine. On the target machine, extract
both archives mymodel.zip
and treelite_runtime.zip
:
john.doe@host-machine:/home/john.doe/$ ssh john.doe@target-machine
Last login: Tue Oct 31 00:43:36 2017 from host-machine
john.doe@target-machine:/home/john.doe/$ unzip mymodel.zip
Archive: mymodel.zip
creating: mymodel/
inflating: mymodel/Makefile
inflating: mymodel/mymodel.c
inflating: mymodel/mymodel.h
inflating: mymodel/recipe.json
john.doe@target-machine:/home/john.doe/$ unzip treelite_runtime.zip
Archive: treelite_runtime.zip
creating: runtime/
.................full output not shown here.................
inflating: runtime/src/common/math.h
6. Now build the runtime package: cmake
followed by make
.
john.doe@target-machine:/home/john.doe/$ cd runtime/build
john.doe@target-machine:/home/john.doe/runtime/build/$ cmake ..
-- The C compiler identification is GNU 7.2.0
-- The CXX compiler identification is GNU 7.2.0
.................full output not shown here.................
-- Configuring done
-- Generating done
-- Build files have been written to: /home/john.doe/runtime/build
john.doe@target-machine:/home/john.doe/runtime/build/$ make
Scanning dependencies of target objtreelite_runtime
[ 50%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/c_api/c_api_common.cc.o
[ 50%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/c_api/c_api_error.cc.o
[ 50%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/c_api/c_api_runtime.cc.o
[ 66%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/logging.cc.o
[ 83%] Building CXX object CMakeFiles/objtreelite_runtime.dir/src/predictor.cc.o
[ 83%] Built target objtreelite_runtime
Scanning dependencies of target treelite_runtime
[100%] Linking CXX shared library ../lib/libtreelite_runtime.so
[100%] Built target treelite_runtime
john.doe@target-machine:/home/john.doe/runtime/build/$ cd ../..
john.doe@target-machine:/home/john.doe/$
Note
Building the runtime package on Windows
The example shown assumes the target was UNIX. On Windows, CMake will create
a Visual Studio solution file named treelite_runtime.sln
. Open it
and compile the solution by selecting Build > Build Solution from the
menu.
Note
Building the runtime package on Mac OS X
The default clang installation on Mac OS X does not support OpenMP, the language construct for multithreading. To enable multithreading in treelite, we recommend that you install gcc 7.x using Homebrew:
john.doe@target-machine:/home/john.doe/runtime/build/$ brew install gcc@7
After g++ is installed, run CMake again with gcc as the C++ compiler:
john.doe@target-machine:/home/john.doe/runtime/build/$ cmake .. \
-DCMAKE_CXX_COMPILER=g++-7 -DCMAKE_C_COMPILER=gcc-7
7. Build the source package. Now only make
will do. (On Windows, run
NMake instead.)
john.doe@target-machine:/home/john.doe/$ cd mymodel
john.doe@target-machine:/home/john.doe/mymodel/$ make
gcc -c -O3 -o mymodel.o mymodel.c -fPIC -std=c99 -flto -fopenmp
gcc -shared -O3 -o mymodel.so mymodel.o -std=c99 -flto -fopenmp
john.doe@target-machine:/home/john.doe/mymodel/$ ls
Makefile mymodel.c mymodel.so
mymodel.h mymodel.o recipe.json
Note
Parallel compilation with GNU Make
If you used parallel_comp
option to split the model into multiple source
files, you can take advantage of parallel compilation. Simply replace make
with make -jN
, where N
is replaced with the number of workers to
launch. Setting N
too high may result into memory shortage.
8. Temporarily set the environment variable PYTHONPATH
to the directory
runtime/python
, so that the runtime package can be found by the Python
interpreter. Then launch an interactive Python session and import the module
treelite.runtime
. If no error occurs, we are done.
john.doe@target-machine:/home/john.doe/$ set PYTHONPATH=./runtime/python/
john.doe@target-machine:/home/john.doe/$ ipython
Python 3.6.2 (default, Jul 17 2017, 16:44:45)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import treelite.runtime
Finally, we are ready to make predictions, per instructions given in
First tutorial. Don’t forget to set PYTHONPATH
before running the
following script:
import treelite.runtime
import numpy as np
# Load data
X = np.load('data.npy')
batch = Batch.from_npy2d(X)
# Load predictor
predictor = treelite.runtime.Predictor('./mymodel/mymodel.so', verbose=True)
out_pred = predictor.predict(batch)
With this option, neither Python nor a C++ compiler is required. You should be able to adopt this option using any basic installation of UNIX-like operating systems.
The target machine shall meet the following conditions:
for
loop; the expf
function in
<math.h>
; the <stdint.h>
header.1. On the host machine, install treelite and import your tree ensemble model.
You should end up with the model object of type Model
.
### Run this block on the **host** machine
import treelite
model = treelite.Model.load('your_model.model', 'xgboost')
# You may also use `from_xgboost` method or the builder class
2. Export your model as a source package by calling the method
export_srcpkg()
of the Model
object. The source package will contain C code representation of the prediction
subroutine.
### Continued from the previous code block
# Operating system of the target machine
platform = 'unix'
# C compiler to use to compile prediction code on the target machine
toolchain = 'gcc'
# Save the source package as a zip archive named mymodel.zip
# Later, we'll use this package to produce the library mymodel.so.
model.export_srcpkg(platform=platform, toolchain=toolchain,
pkgpath='./mymodel.zip', libname='mymodel.so',
verbose=True)
Note
On the value of toolchain
Treelite supports only three toolchain configurations (‘msvc’, ‘gcc’, ‘clang’)
for which it generates Makefiles. If you are using a compiler other than
these three, you will have to write your own Makefile. For now, just set
toolchain='gcc'
and move on.
After calling export_srcpkg()
, you should be able to
find the zip archive named mymodel.zip
inside the current working directory.
john.doe@host-machine:/home/john.doe/$ ls .
mymodel.zip your_model.model
The content of mymodel.zip
consists of the header and source files, as well
as the Makefile:
john.doe@host-machine:/home/john.doe/$ unzip -l mymodel.zip
Archive: mymodel.zip
Length Date Time Name
--------- ---------- ----- ----
0 11-01-2017 23:11 mymodel/
167 11-01-2017 23:11 mymodel/Makefile
4831036 11-01-2017 23:11 mymodel/mymodel.c
311 11-01-2017 23:11 mymodel/mymodel.h
109 11-01-2017 23:11 mymodel/recipe.json
--------- -------
4831623 5 files
3. Now you are ready to deploy the model to the target machine. Copy to the
target machine the archive mymodel.zip
(source package).
john.doe@host-machine:/home/john.doe/$ sftp john.doe@target-machine
Connected to target-machine.
sftp> put mymodel.zip
Uploading mymodel.zip to /home/john.doe/mymodel.zip
mymodel.zip 100% 410KB 618.2KB/s 00:00
sftp> quit
4. It is time to move to the target machine. On the target machine, extract
the archive mymodel.zip
:
john.doe@host-machine:/home/john.doe/$ ssh john.doe@target-machine
Last login: Tue Oct 31 00:43:36 2017 from host-machine
john.doe@target-machine:/home/john.doe/$ unzip mymodel.zip
Archive: mymodel.zip
creating: mymodel/
inflating: mymodel/Makefile
inflating: mymodel/mymodel.c
inflating: mymodel/mymodel.h
inflating: mymodel/recipe.json
5. Build the source package (using GNU Make or NMake).
john.doe@target-machine:/home/john.doe/$ cd mymodel
john.doe@target-machine:/home/john.doe/mymodel/$ make
gcc -c -O3 -o mymodel.o mymodel.c -fPIC -std=c99 -flto -fopenmp
gcc -shared -O3 -o mymodel.so mymodel.o -std=c99 -flto -fopenmp
john.doe@target-machine:/home/john.doe/mymodel/$ ls
Makefile mymodel.c mymodel.so
mymodel.h mymodel.o recipe.json
Note
Parallel compilation with GNU Make
If you used parallel_comp
option to split the model into multiple source
files, you can take advantage of parallel compilation. Simply replace make
with make -jN
, where N
is replaced with the number of workers to
launch. Setting N
too high may result into memory shortage.
Note
Using other compilers
If you are using a compiler other than gcc, clang, or Microsoft Visual C++,
you will need to compose your own Makefile. Open the Makefile
and
make necessary changes.
The prediction library provides the function predict
with the
following signature:
float predict(union Entry* data, int pred_margin);
Here, the argument data
must be an array of length M
, where M
is
the number of features used in the tree ensemble. The data
array stores
all the feature values of a single row. To indicate presence or absence of
a feature value, we use the union type Entry
, which defined as
union Entry {
int missing;
float fvalue;
};
For missing values, we set the missing
field to -1. For non-missing ones, we
set the fvalue
field to the feature value. The total number of features
is given by the function
size_t get_num_feature(void);
Let’s look at an example. We’d start by initializing the array inst
, a dense
aray to hold feature values of a single data row:
/* number of features */
const size_t num_feature = get_num_feature();
/* inst: dense vector storing feature values */
union Entry* inst = malloc(sizeof(union Entry) * num_feature);
/* clear inst with all missing values */
for (i = 0; i < num_feature; ++i) {
inst[i].missing = -1;
}
Before calling the function predict
, the array inst
needs to be
initialized with missing and present feature values. The following peudocode
illustrates the idea:
For each data row rid:
inst[i].missing == -1 for every i, assuming all features lack values
For each feature i for which the data row in fact has a feature value:
Set inst[i].fvalue = [feature value], to indicate presence
Call predict(inst, 0) and get prediction for the data row rid
For each feature i for which the row has a feature value:
Set inst[i].missing = -1, to prepare for next row (rid + 1)
The task is not too difficult as long as the input data is given as a particular form of sparse matrix: the Compressed Sparse Row format. The sparse matrix consists of three arrays:
val
stores nonzero entries in
row-major order.col_ind
stores column indices of the entries in val
. The expression
col_ind[i]
indicates the column index of the i
th entry val[i]
.row_ptr
stores the locations in val
that start and end data rows. The
i
th data row is given by the array slice val[row_ptr[i]:row_ptr[i+1]]
./* nrow : number of data rows */
for (rid = 0; rid < nrow; ++rid) {
ibegin = row_ptr[rid];
iend = row_ptr[rid + 1];
/* Fill nonzeros */
for (i = ibegin; i < iend; ++i) {
inst[col_ind[i]].fvalue = val[i];
}
out_pred[rid] = predict(inst, 0);
/* Drop nonzeros */
for (i = ibegin; i < iend; ++i) {
inst[col_ind[i]].missing = -1;
}
}
It only remains to create three arrays val
, col_ind
, and row_ptr
.
You may want to use a third-pary library here to read from
a SVMLight format. For now, we’ll punt the issue of loading the input data
and write it out as constants in the program:
#include <stdio.h>
#include <stdlib.h>
#include "mymodel.h"
int main(void) {
/* 5x13 "sparse" matrix, in CSR format
[[ 0. , 0. , 0.68, 0.99, 0. , 0.11, 0. , 0.82, 0. ,
0. , 0. , 0. , 0. ],
[ 0. , 0. , 0.99, 0. , 0. , 0. , 0. , 0. , 0. ,
0.61, 0. , 0. , 0. ],
[ 0.02, 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ],
[ 0. , 0. , 0.36, 0. , 0.82, 0. , 0. , 0.57, 0. ,
0. , 0. , 0. , 0.75],
[ 0.47, 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0.45, 0. ]]
*/
const float val[] = {0.68, 0.99, 0.11, 0.82, 0.99, 0.61, 0.02, 0.36, 0.82,
0.57, 0.75, 0.47, 0.45};
const size_t col_ind[] = {2, 3, 5, 7, 2, 9, 0, 2, 4, 7, 12, 0, 11};
const size_t row_ptr[] = {0, 4, 6, 7, 11, 13};
const size_t nrow = 5;
const size_t ncol = 13;
/* number of features */
const size_t num_feature = get_num_feature();
/* inst: dense vector storing feature values */
union Entry* inst = malloc(sizeof(union Entry) * num_feature);
float* out_pred = malloc(sizeof(float) * nrow);
size_t rid, ibegin, iend, i;
/* clear inst with all missing */
for (i = 0; i < num_feature; ++i) {
inst[i].missing = -1;
}
for (rid = 0; rid < nrow; ++rid) {
ibegin = row_ptr[rid];
iend = row_ptr[rid + 1];
/* Fill nonzeros */
for (i = ibegin; i < iend; ++i) {
inst[col_ind[i]].fvalue = val[i];
}
out_pred[rid] = predict(inst, 0);
/* Drop nonzeros */
for (i = ibegin; i < iend; ++i) {
inst[col_ind[i]].missing = -1;
}
printf("pred[%zu] = %f\n", rid, out_pred[rid]);
}
free(inst);
free(out_pred);
return 0;
}
Save the program as a .c file and put it in the same directory mymodel/
. To
link the program against the prediction library mymodel.so
, simply run
gcc -o myprog myprog.c mymodel.so -I. -std=c99
As long as the program myprog
is in the same directory of the prediction
library mymodel.so
, we’ll be good to go.
A sample output:
pred[0] = 44.880001
pred[1] = 44.880001
pred[2] = 44.880001
pred[3] = 42.670002
pred[4] = 44.880001
Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.
Using XGBoost or LightGBM for training? Read this document instead.
Contents
The ModelBuilder
class is a tool used to specify decision
tree ensembles programmatically. Each tree ensemble is represented as follows:
Tree
object is a dictionary of
nodes indexed by unique integer keys.ModelBuilder
object is a list of
Tree
objects.Consider the following tree ensemble, consisting of two regression trees:
Note
Provision for missing data: default directions
Decision trees in treelite accomodate missing data by indicating the default direction for every test node. In the diagram above, the default direction is indicated by label “Missing.” For instance, the root node of the first tree shown above will send to the left all data points that lack values for feature 0.
For now, let’s assume that we’ve somehow found optimal choices of default directions at training time. For detailed instructions for actually deciding default directions, see Section 3.4 of the XGBoost paper.
Let us construct this ensemble using the model builder. First step is to assign unique integer key to each node. In the following diagram, integer keys are indicated in red. Note that integer keys need to be unique only within the same tree.
Next, we create a model builder object by calling the constructor for
ModelBuilder
, with an num_feature
argument indicating
the total number of features used in the ensemble:
import treelite
builder = treelite.ModelBuilder(num_feature=3)
We also create a tree object; it will represent the first tree in the ensemble.
# to represent the first tree
tree = treelite.ModelBuilder.Tree()
The first tree has five nodes, each of which is to be inserted into the tree one at a time. The syntax for node insertion is as follows:
tree[0] # insert a new node with key 0
Once a node has been inserted, we can refer to it by writing
tree[0] # refer to existing node #0
The meaning of the expression tree[0]
thus depends on whether the node #0
exists in the tree or not.
We may combine node insertion with a function call to specify its content.
For instance, node #0 is a test node, so we call
set_numerical_test_node()
:
# Node #0: feature 0 < 5.0 ? (default direction left)
tree[0].set_numerical_test_node(feature_id=0,
opname='<',
threshold=5.0,
default_left=True,
left_child_key=1,
right_child_key=2)
On the other hand, node #2 is a leaf node, so call
set_leaf_node()
instead:
# Node #2: leaf with output +0.6
tree[2].set_leaf_node(0.6)
Let’s go ahead and specify the other three nodes:
# Node #1: feature 2 < -3.0 ? (default direction right)
tree[1].set_numerical_test_node(feature_id=2,
opname='<',
threshold=-3.0,
default_left=False,
left_child_key=3,
right_child_key=4)
# Node #3: leaf with output -0.4
tree[3].set_leaf_node(-0.4)
# Node #4: leaf with output +1.2
tree[4].set_leaf_node(1.2)
We must indicate which node is the root:
# Set node #0 as root
tree[0].set_root()
We are now done with the first tree. We insert it with the model builder
by calling append()
. (Recall that the model
builder is really a list of tree objects, hence the method name append
.)
# Insert the first tree into the ensemble
builder.append(tree)
The second tree is constructed analogously:
tree2 = treelite.ModelBuilder.Tree()
# Node #0: feature 1 < 2.5 ? (default direction right)
tree2[0].set_numerical_test_node(feature_id=1,
opname='<',
threshold=2.5,
default_left=False,
left_child_key=1,
right_child_key=2)
# Set node #0 as root
tree2[0].set_root()
# Node #1: leaf with output +1.6
tree2[1].set_leaf_node(1.6)
# Node #2: feature 2 < -1.2 ? (default direction left)
tree2[2].set_numerical_test_node(feature_id=2,
opname='<',
threshold=-1.2,
default_left=True,
left_child_key=3,
right_child_key=4)
# Node #3: leaf with output +0.1
tree2[3].set_leaf_node(0.1)
# Node #4: leaf with output -0.3
tree2[4].set_leaf_node(-0.3)
# Insert the second tree into the ensemble
builder.append(tree2)
We are now done building the member trees. The last step is to call
commit()
to finalize the ensemble into
a Model
object:
# Finalize and obtain Model object
model = builder.commit()
Note
Difference between ModelBuilder
and
Model
objects
Why does treelite require one last step of “committing”? All
Model
objects are immutable; once constructed,
they cannot be modified at all. So you won’t be able to add a tree or a node
to an existing Model
object, for instance. On the other
hand, ModelBuilder
objects are mutable, so that you
can iteratively build trees.
To ensure we got all details right, we can examine the resulting C program.
model.compile(dirpath='./test')
with open('./test/test.c', 'r') as f:
for line in f.readlines():
print(line, end='')
which produces the output
/* Other functions omitted for space consideration */
float predict_margin(union Entry* data) {
float sum = 0.0f;
if (!(data[0].missing != -1) || data[0].fvalue < 5) {
if ( (data[2].missing != -1) && data[2].fvalue < -3) {
sum += (float)-0.4;
} else {
sum += (float)1.2;
}
} else {
sum += (float)0.6;
}
if ( (data[1].missing != -1) && data[1].fvalue < 2.5) {
sum += (float)1.6;
} else {
if (!(data[2].missing != -1) || data[2].fvalue < -1.2) {
sum += (float)0.1;
} else {
sum += (float)-0.3;
}
}
return sum + (0);
}
The toy example has been helpful as an illustration, but it is impractical to manually specify nodes for real-world ensemble models. The following section will show us how to automate the tree building process. We will look at scikit-learn in particular.
Scikit-learn (scikit-learn/scikit-learn) is a Python machine learning package known for its versatility and ease of use. It supports a wide variety of models and algorithms.
Treelite will be able to work with any decision tree ensemble models produced by scikit-learn. In particular, it will be able to work with
sklearn.ensemble.RandomForestRegressor
sklearn.ensemble.RandomForestClassifier
sklearn.ensemble.GradientBoostingRegressor
sklearn.ensemble.GradientBoostingClassifier
Note
Why scikit-learn? How about other packages?
We had to pick a specific example for programmatic tree construction, so we chose scikit-learn. If you’re using another package, don’t lose heart. As you read through the rest of section, notice how specific pieces of information about the tree ensemble model are being extracted. As long as your choice of package exposes equivalent information, you’ll be able to adapt the example to your needs.
Note
In a hurry? Try the gallery module
The rest of this document explains in detail how to import scikit-learn
models using the builder class. If you prefer to skip all the gory details,
simply import the module treelite.gallery.sklearn
.
import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)
Note
Adaboost ensembles not yet supported
Treelite currently does not support weighting of member trees, so you won’t be able to use Adaboost ensembles.
Let’s start with the Boston house prices dataset, a regression problem. (Classification problems are somewhat trickier, so we’ll save them for later.)
We’ll be using RandomForestRegressor
, a random
forest for regression. A random forest is an ensemble of decision trees
that are independently trained on random samples from the training data. See
this page for
more details. For now, just remember to specify random_forest=True
in the
ModelBuilder
constructor.
import sklearn.datasets
import sklearn.ensemble
# Load the Boston housing dataset
X, y = sklearn.datasets.load_boston(return_X_y=True)
# Train a random forest regressor with 10 trees
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)
We shall programmatically construct Tree
objects from internal attributes of the scikit-learn model. We only need
to define a few helper functions.
For the rest of sections, we’ll be diving into lots of details that are specific to scikit-learn. Many details have been adopted from this reference page.
The function process_model() takes in a scikit-learn ensemble object and
returns the completed Model
object:
def process_model(sklearn_model):
# Initialize treelite model builder
# Set random_forest=True for random forests
builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
random_forest=True)
# Iterate over individual trees
for i in range(sklearn_model.n_estimators):
# Process the i-th tree and add to the builder
# process_tree() to be defined later
builder.append( process_tree(sklearn_model.estimators_[i].tree_,
sklearn_model) )
return builder.commit()
The usage of this function is as follows:
model = process_model(clf)
We won’t have space here to discuss all internals of scikit-learn objects, but a few details should be noted:
n_features_
stores the number of features used anywhere
in the tree ensemble.n_estimators
stores the number of member trees.estimators_
is an array of handles that store the individual
member trees. To access the object for the i
-th tree, write
estimators_[i].tree_
. This object will be passed to the function
process_tree()
.The function process_tree() takes in a single scikit-learn tree object
and returns an object of type Tree
:
def process_tree(sklearn_tree, sklearn_model):
treelite_tree = treelite.ModelBuilder.Tree()
# Node #0 is always root for scikit-learn decision trees
treelite_tree[0].set_root()
# Iterate over each node: node ID ranges from 0 to [node_count]-1
for node_id in range(sklearn_tree.node_count):
process_node(treelite_tree, sklearn_tree, node_id, sklearn_model)
return treelite_tree
Explanations:
node_count
stores the number of nodes in the decision tree.[node_count]-1
.The function process_node() determines whether each node is a leaf node
or a test node. It does so by looking at the attribute children_left
:
If the left child of the node is set to -1, that node is thought to be
a leaf node.
def process_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
if sklearn_tree.children_left[node_id] == -1: # leaf node
process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model)
else: # test node
process_test_node(treelite_tree, sklearn_tree, node_id, sklearn_model)
The function process_test_node() extracts the content of a test node
and passes it to the Tree
object that is
being constructed.
def process_test_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
# Initialize the test node with given node ID
treelite_tree[node_id].set_numerical_test_node(
feature_id=sklearn_tree.feature[node_id],
opname='<=',
threshold=sklearn_tree.threshold[node_id],
default_left=True,
left_child_key=sklearn_tree.children_left[node_id],
right_child_key=sklearn_tree.children_right[node_id])
Explanations:
feature
is the array containing feature indices used
in test nodes.threshold
is the array containing threshold values used
in test nodes.[feature value] <= [threshold]
.children_left
and children_right
together store
children’s IDs for test nodes.Note
Scikit-learn and missing data
Scikit-learn handles missing data differently than XGBoost and treelite.
Before training an ensemble model, you’ll have to impute
missing values. For this reason, test nodes in scikit-learn tree models will
contain no “default direction.” We will assign default_left=True
arbitrarily for test nodes to keep treelite happy.
The function process_leaf_node() defines a leaf node:
def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
# The `value` attribute stores the output for every leaf node.
leaf_value = sklearn_tree.value[node_id].squeeze()
# Initialize the leaf node with given node ID
treelite_tree[node_id].set_leaf_node(leaf_value)
Let’s test it out:
model = process_model(clf)
model.export_lib(libpath='./libtest.dylib', toolchain='gcc', verbose=True)
import treelite.runtime
predictor = treelite.runtime.Predictor(libpath='./libtest.dylib')
predictor.predict(treelite.runtime.Batch.from_npy2d(X))
Gradient boosting is an algorithm where decision trees are trained one at a
time, ensuring that latter trees complement former trees. See this page
for more details. Treelite makes distinction between random forests and
gradient boosted trees by the value of random_forest
flag in the
ModelBuilder
constructor.
Note
Set init='zero'
to ensure compatibility
To make sure that the gradient boosted model is compatible with treelite,
make sure to set init='zero'
in the
GradientBoostingRegressor
constructor. This
ensures that the compiled prediction subroutine will produce the correct
prediction output. Gradient boosting models trained without specifying
init='zero'
in the constructor are NOT supported by treelite!
# Gradient boosting regressor
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingRegressor(n_estimators=10,
init='zero')
clf.fit(X, y)
We will recycle most of the helper code we wrote earlier. Only two functions will need to be modified:
# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions
def process_model(sklearn_model):
# Check for init='zero'
if sklearn_model.init != 'zero':
raise Exception("Gradient boosted trees must be trained with "
"the option init='zero'")
# Initialize treelite model builder
# Set random_forest=False for gradient boosted trees
builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
random_forest=False)
for i in range(sklearn_model.n_estimators):
# Process i-th tree and add to the builder
builder.append( process_tree(sklearn_model.estimators_[i][0].tree_,
sklearn_model) )
return builder.commit()
def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
leaf_value = sklearn_tree.value[node_id].squeeze()
# Need to shrink each leaf output by the learning rate
leaf_value *= sklearn_model.learning_rate
# Initialize the leaf node with given node ID
treelite_tree[node_id].set_leaf_node(leaf_value)
Some details specific to GradientBoostingRegressor
:
random_forest=False
in the ModelBuilder
constructor.estimators_[i][0].tree_
, as estimators_[i]
returns an array consisting
of a single tree (i
-th tree).Let’s test it:
# Convert to treelite model
model = process_model(clf)
# Generate shared library
model.export_lib(libpath='./libtest2.dylib', toolchain='gcc', verbose=True)
# Make prediction with predictor
predictor = treelite.runtime.Predictor(libpath='./libtest2.dylib')
predictor.predict(treelite.runtime.Batch.from_npy2d(X))
For binary classification, let’s use the digits dataset. We will take 0’s and 1’s from the dataset and treat 0’s as the negative class and 1’s as the positive.
# load a binary classification problem
# Set n_class=2 to produce two classes
digits = sklearn.datasets.load_digits(n_class=2)
X, y = digits['data'], digits['target']
# Should print [0 1]
print(np.unique(y))
# Train a random forest classifier
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)
Random forest classifiers in scikit-learn store frequency counts for the positive and negative class. For instance, a leaf node may output a set of counts
[ 100, 200 ]
which indicates the following:
Again, most of the helper functions may be re-used; only two functions need to be rewritten. Explanation will follow after the code:
# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions
def process_model(sklearn_model):
builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
random_forest=True)
for i in range(sklearn_model.n_estimators):
# Process i-th tree and add to the builder
builder.append( process_tree(sklearn_model.estimators_[i].tree_,
sklearn_model) )
return builder.commit()
def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
# Get counts for each label (+/-) at this leaf node
leaf_count = sklearn_tree.value[node_id].squeeze()
# Compute the fraction of positive data points at this leaf node
fraction_positive = float(leaf_count[1]) / leaf_count.sum()
# The fraction above is now the leaf output
treelite_tree[node_id].set_leaf_node(fraction_positive)
As noted earlier, we access the frequency counts at each leaf node, reading the
value
attribute of each tree. Then we compute the fraction of positive
data points with respect to all training data points belonging to the leaf.
This fraction then becomes the leaf output. This way, leaf nodes now produce
single numbers rather than frequency count arrays.
Why did we have to compute a fraction? For binary classification, treelite expects each tree to produce a single number output. At prediction time, the outputs from the member trees will get averaged to produce the final prediction, which is also a single number. By setting the positive fraction as the leaf output, we ensure that the final prediction is a proper probability value. For instance, if an ensemble consisting of 5 trees produces the following set of outputs
Tree 0 0.1
Tree 1 0.7
Tree 2 0.4
Tree 3 0.3
Tree 4 0.7
then the final prediction will be 0.44, which we interpret as 44% probability for the positive class.
Let’s use the digits dataset again, this time with 4 classes (i.e. 0’s, 1’s, 2’s, and 3’s).
# Load a multi-class classification problem
# Set n_class=4 to produce four classes
digits = sklearn.datasets.load_digits(n_class=4)
X, y = digits['data'], digits['target']
# Should print [0 1 2 3]
print(np.unique(y))
# Train a random forest classifier
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)
Random forest classifiers in scikit-learn store frequency counts (see the explanation in the previous section). For instance, a leaf node may output a set of counts
[ 100, 400, 300, 200 ]
which shows that the total of 1000 training data points belong to this leaf node and that 100, 400, 300, and 200 of them are labeled class 0, 1, 2, and 3, respectively.
We will have to re-write the process_leaf_node() function to accomodate multiple classes.
# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions
def process_model(sklearn_model):
# Must specify num_output_group and pred_transform
builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
num_output_group=sklearn_model.n_classes_,
random_forest=True,
pred_transform='identity_multiclass')
for i in range(sklearn_model.n_estimators):
# Process i-th tree and add to the builder
builder.append( process_tree(sklearn_model.estimators_[i].tree_,
sklearn_model) )
return builder.commit()
def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
# Get counts for each label class at this leaf node
leaf_count = sklearn_tree.value[node_id].squeeze()
# Compute the probability distribution over label classes
prob_distribution = leaf_count / leaf_count.sum()
# The leaf output is the probability distribution
treelite_tree[node_id].set_leaf_node(prob_distribution)
The process_leaf_node()
function is quite similar to what we had for the
binary classification case. Only difference is that, instead of computing the
fraction of the positive class, we compute the probability distribution for
all possible classes. Each leaf node thus will store the probability
distribution of possible class outcomes.
The process_model()
function is also similar to what we had before. The
crucial difference is the existence of parameters num_output_group
and
pred_transform
. The num_output_group
parameter is used only for
multi-class classification: it should store the number of classes (in this
example, 4). The pred_transform
parameter should be set to
'identity_multiclass'
, to indicate
that the prediction should be made simply by averaging the probability
distribution produced by each leaf node. (Leaf outputs are averaged rather
than summed because we set random_forest=True
.) For instance, if an ensemble
consisting of 3 trees produces the following set of outputs
Tree 0 [ 0.5, 0.5, 0.0, 0.0 ]
Tree 1 [ 0.1, 0.5, 0.3, 0.1 ]
Tree 2 [ 0.2, 0.5, 0.2, 0.1 ]
then the final prediction will be the average
[ 0.26666667, 0.5, 0.16666667, 0.06666667 ]
, which indicates 26.7%
probability for the first class, 50.0% for the second, 16.7% for the third,
and 6.7% for the fourth.
We use the digits dataset. We will take 0’s and 1’s from the dataset and treat 0’s as the negative class and 1’s as the positive.
# Load a binary classification problem
# Set n_class=2 to produce two classes
digits = sklearn.datasets.load_digits(n_class=2)
X, y = digits['data'], digits['target']
# Should print [0 1]
print(np.unique(y))
# Train a gradient boosting classifier
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingClassifier(n_estimators=10,
init='zero')
clf.fit(X, y)
Note
Set init='zero'
to ensure compatibility
To make sure that the gradient boosted model is compatible with treelite,
make sure to set init='zero'
in the
GradientBoostingClassifier
constructor. This
ensures that the compiled prediction subroutine will produce the correct
prediction output. Gradient boosting models trained without specifying
init='zero'
in the constructor are NOT supported by treelite!
Here are the functions process_model()
and process_leaf_node()
for this
scenario:
# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions
def process_model(sklearn_model):
# Check for init='zero'
if sklearn_model.init != 'zero':
raise Exception("Gradient boosted trees must be trained with "
"the option init='zero'")
# Initialize treelite model builder
# Set random_forest=False for gradient boosted trees
# Set pred_transform='sigmoid' to obtain probability predictions
builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
random_forest=False,
pred_transform='sigmoid')
for i in range(sklearn_model.n_estimators):
# Process i-th tree and add to the builder
builder.append( process_tree(sklearn_model.estimators_[i][0].tree_,
sklearn_model) )
return builder.commit()
def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
leaf_value = sklearn_tree.value[node_id].squeeze()
# Need to shrink each leaf output by the learning rate
leaf_value *= sklearn_model.learning_rate
# Initialize the leaf node with given node ID
treelite_tree[node_id].set_leaf_node(leaf_value)
Some details specific to GradientBoostingClassifier
:
random_forest=False
in the ModelBuilder
constructor.estimators_[i][0].tree_
, as estimators_[i]
returns an array consisting
of a single tree (i
-th tree).In addition, we specify the parameter pred_transform='sigmoid'
so that
the final prediction yields the probability for the positive class. For example,
suppose that an ensemble consisting of 4 trees produces the following set of
outputs:
Tree 0 +0.5
Tree 1 -2.3
Tree 2 +1.5
Tree 3 -1.5
Unlike the random forest example earlier, we do not assume that each leaf output is between 0 and 1; it can be any real number, negative or positive. These numbers are referred to as margin scores, to distinguish them from probabilities.
To obtain the probability for the positive class, we first sum the margin scores (outputs) from the member trees.
Tree 0 +0.5
Tree 1 -2.3
Tree 2 +1.5
Tree 3 -1.5
--------------
Total -1.8
Then we apply the sigmoid function:
The resulting value is the final prediction. You may interpret this value as a probability. For the particular example, the sigmoid value of -1.8 is 0.14185106, which we interpret as 14.2% probability for the positive class.
Let’s use the digits dataset again, this time with 4 classes (i.e. 0’s, 1’s, 2’s, and 3’s).
# Load a multi-class classification problem
# Set n_class=4 to produce four classes
digits = sklearn.datasets.load_digits(n_class=4)
X, y = digits['data'], digits['target']
# Should print [0 1 2 3]
print(np.unique(y))
# Train a gradient boosting classifier
# Notice the argument init='zero'
clf = sklearn.ensemble.GradientBoostingClassifier(n_estimators=10,
init='zero')
clf = sklearn.ensemble.RandomForestClassifier(n_estimators=10)
clf.fit(X, y)
Note
Set init='zero'
to ensure compatibility
To make sure that the gradient boosted model is compatible with treelite,
make sure to set init='zero'
in the
GradientBoostingClassifier
constructor. This
ensures that the compiled prediction subroutine will produce the correct
prediction output. Gradient boosting models trained without specifying
init='zero'
in the constructor are NOT supported by treelite!
Here are the functions process_model()
and process_leaf_node()
for this
scenario:
# process_tree(), process_node(), process_test_node() omitted to save space
# See the first section for their definitions
def process_model(sklearn_model):
# Check for init='zero'
if sklearn_model.init != 'zero':
raise Exception("Gradient boosted trees must be trained with "
"the option init='zero'")
# Initialize treelite model builder
# Set random_forest=False for gradient boosted trees
# Set num_output_group for multiclass classification
# Set pred_transform='softmax' to obtain probability predictions
builder = treelite.ModelBuilder(num_feature=sklearn_model.n_features_,
num_output_group=sklearn_model.n_classes_,
random_forest=False,
pred_transform='softmax')
# Process [number of iterations] * [number of classes] trees
for i in range(sklearn_model.n_estimators):
for k in range(sklearn_model.n_classes_):
builder.append( process_tree(sklearn_model.estimators_[i][k].tree_,
sklearn_model) )
return builder.commit()
def process_leaf_node(treelite_tree, sklearn_tree, node_id, sklearn_model):
leaf_value = sklearn_tree.value[node_id].squeeze()
# Need to shrink each leaf output by the learning rate
leaf_value *= sklearn_model.learning_rate
# Initialize the leaf node with given node ID
treelite_tree[node_id].set_leaf_node(leaf_value)
The process_leaf_node()
function is identical to one in the previous
section: as before, each leaf node produces a single real-number output.
On the other hand, the process_model()
function needs some explanation.
First of all, the attribute estimators_
of the scikit-learn model object
now stores output groups, which are simply groups of decision trees.
The expression estimators_[i]
thus refers to the i
th output group.
Each output group contains as many trees as there are label classes. For the
digits example with 4 label classes, we’d have 4 trees for each output group:
estimators_[i][0]
, estimators_[i][1]
, estimators_[i][2]
, and
estimators_[i][3]
. Since there are as many output groups as the number of
iterations used for training, the total number of member trees is
[number of iterations] * [number of classes]
. We have to call append()
once for each member tree; hence the use of nested loop.
We also set pred_transform='softmax'
, which indicates the way margin
outputs should be transformed to produce probability predictions. Let us look
at a concrete example: suppose we train an ensemble model with 3 rounds of
gradient boosting. It would produce a total of 12 decision trees (3 rounds *
4 classes). Suppose also that, given a single test data point, the model
produces the following set of margins:
Output group 0:
Tree 0 produces +0.5
Tree 1 produces +1.5
Tree 2 produces -2.3
Tree 3 produces -1.5
Output group 1:
Tree 4 produces +0.1
Tree 5 produces +0.7
Tree 6 produces +1.5
Tree 7 produces -0.9
Output group 2:
Tree 8 produces -0.1
Tree 9 produces +0.3
Tree 10 produces -0.7
Tree 11 produces +0.2
How do we compute probabilities for each of the 4 classes? First, we compute the sum of the margin scores for each output group:
Output group 0:
Tree 0 produces +0.5
Tree 1 produces +1.5
Tree 2 produces -2.3
Tree 3 produces -1.5
----------------------
SUBTOTAL -1.8
Output group 1:
Tree 4 produces +0.1
Tree 5 produces +0.7
Tree 6 produces +1.5
Tree 7 produces -0.9
----------------------
SUBTOTAL +1.4
Output group 2:
Tree 8 produces -0.1
Tree 9 produces +0.3
Tree 10 produces -0.7
Tree 11 produces +0.2
----------------------
SUBTOTAL -0.3
The vector [-1.8, +1.4, -0.3]
consisting of the subtotals quantifies the
relative likelihood of the label classes. Since the second element (1.4) is
the largest, the second class must be the most likely outcome for the particular
data point. This vector is not yet a probability distribution, since its
elements do not sum to 1.
The softmax function transforms any real-valued vector into a probability distribution as follows:
exp
) to every element in the vector.
This step ensures that every element is positive.Let’s walk through the steps with the vector [-1.8, +1.4, -0.3]
. Applying
the exponential function is simple with Python:
x = np.exp([-1.8, +1.4, -0.3])
print(x)
which yields
[ 0.16529889 4.05519997 0.74081822]
Note that every element is now positive. Then we normalize the vector by writing
x = x / x.sum()
print(x)
which gives a proper probability distribution:
[ 0.03331754 0.8173636 0.14931886]
We can now interpret the result as giving 3.3% probability for the first class, 81.7% probability for the second, and 14.9% probability for the third.
Since the scope of treelite is limited to prediction only, one must use other machine learning packages to train decision tree ensemble models. In this document, we will show how to import an ensemble model that had been trained elsewhere.
Using XGBoost or LightGBM for training? Read this document instead.
Contents
Protocol Buffers (google/protobuf) is a widely used mechanism to serialize structured data. You may specify your ensemble model according to the specification src/tree.proto. Depending on the package you used to train the model, it may take some effort to express the model in terms of the given spec. See this helpful guide on reading and writing serialized messages.
To import models that had been serialized with Protocol Buffers,
use the load()
method with argument
format='protobuf'
:
# model had been saved to a file named my_model.bin
# notice the second argument format='protobuf'
model = Model.load('my_model.bin', format='protobuf')
API of treelite Python package.
treelite: a framework to optimize decision tree ensembles for fast prediction
treelite.
DMatrix
(data, data_format=None, missing=None, feature_names=None, feature_types=None, verbose=False, nthread=None)¶Data matrix used in treelite.
Parameters: |
|
---|
treelite.
Model
(handle=None)¶Decision tree ensemble model
Parameters: | handle (ctypes.c_void_p , optional) – Initial value of model handle |
---|
compile
(dirpath, params=None, compiler='ast_native', verbose=False)¶Generate prediction code from a tree ensemble model. The code will be C99
compliant. One header file (.h) will be generated, along with one or more
source files (.c). Use create_shared()
method to package
prediction code as a dynamic shared library (.so/.dll/.dylib).
Parameters: |
---|
Example
The following populates the directory ./model
with source and header
files:
model.compile(dirpath='./my/model', params={}, verbose=True)
If parallel compilation is enabled (parameter parallel_comp
), the files
are in the form of ./my/model/header.h
, ./my/model/main.c
,
./my/model/tu0.c
, ./my/model/tu1.c
and so forth, depending on
the value of parallel_comp
. Otherwise, there will be exactly two files:
./model/header.h
, ./my/model/main.c
export_as_xgboost
(filename, name_obj)¶(EXPERIMENTAL FEATURE) Export a tree ensemble model as a XGBoost model file
Parameters: |
|
---|
Example
model.export_as_xgboost('xgboost_model.model')
export_lib
(toolchain, libpath, params=None, compiler='ast_native', verbose=False, nthread=None, options=None)¶Convenience function: Generate prediction code and immediately turn it into a dynamic shared library. A temporary directory will be created to hold the source files.
Parameters: |
|
---|
Example
The one-line command
model.export_lib(toolchain='msvc', libpath='./mymodel.dll',
params={}, verbose=True)
is equivalent to the following sequence of commands:
model.compile(dirpath='/temporary/directory', params={}, verbose=True)
create_shared(toolchain='msvc', dirpath='/temporary/directory',
verbose=True)
# move the library out of the temporary directory
shutil.move('/temporary/directory/mymodel.dll', './mymodel.dll')
export_srcpkg
(platform, toolchain, pkgpath, libname, params=None, compiler='ast_native', verbose=False, options=None)¶Convenience function: Generate prediction code and create a zipped source package for deployment. The resulting zip file will also contain a Makefile.
Parameters: |
|
---|
Example
The one-line command
model.export_srcpkg(platform='unix', toolchain='gcc',
pkgpath='./mymodel_pkg.zip', libname='mymodel.so',
params={}, verbose=True)
is equivalent to the following sequence of commands:
model.compile(dirpath='/temporary/directory/mymodel',
params={}, verbose=True)
generate_makefile(dirpath='/temporary/directory/mymodel',
platform='unix', toolchain='gcc')
# zip the directory containing C code and Makefile
shutil.make_archive(base_name=pkgpath, format='zip',
root_dir='/temporary/directory',
base_dir='mymodel/')
from_xgboost
(booster)¶Load a tree ensemble model from an XGBoost Booster object
Parameters: | booster (object of type xgboost.Booster ) – Python handle to XGBoost model |
---|---|
Returns: | model – loaded model |
Return type: | Model object |
Example
bst = xgboost.train(params, dtrain, 10, [(dtrain, 'train')])
xgb_model = Model.from_xgboost(bst)
treelite.
ModelBuilder
(num_feature, num_output_group=1, random_forest=False, **kwargs)¶Builder class for tree ensemble model: provides tools to iteratively build an ensemble of decision trees
Parameters: |
|
---|
Node
¶Handle to a node in a tree
set_categorical_test_node
(feature_id, left_categories, default_left, left_child_key, right_child_key)¶Set the node as a test node with categorical split. A list defines all
categories that would be classified as the left side. Categories are
integers ranging from 0
to n-1
, where n
is the number of
categories in that particular feature.
Parameters: |
|
---|
set_leaf_node
(leaf_value)¶Set the node as a leaf node
Parameters: | leaf_value (float / list of float ) – Usually a single leaf value (weight) of the leaf node. For multiclass
random forest classifier, leaf_value should be a list of leaf weights. |
---|
set_numerical_test_node
(feature_id, opname, threshold, default_left, left_child_key, right_child_key)¶Set the node as a test node with numerical split. The test is in the form
[feature value] OP [threshold]
. Depending on the result of the test,
either left or right child would be taken.
Parameters: |
|
---|
set_root
()¶Set the node as the root
Tree
¶Handle to a decision tree in a tree ensemble Builder
append
(tree)¶Add a tree at the end of the ensemble
Parameters: | tree (Tree object) – tree to be added |
---|
Example
builder = ModelBuilder(num_feature=4227)
tree = ... # build tree somehow
builder.append(tree) # add tree at the end of the ensemble
commit
()¶Finalize the ensemble model
Returns: | model – finished model |
---|---|
Return type: | Model object |
Example
builder = ModelBuilder(num_feature=4227)
for i in range(100):
tree = ... # build tree somehow
builder.append(tree) # add one tree at a time
model = builder.commit() # now get a Model object
model.compile(dirpath='test') # compile model into C code
insert
(tree, index)¶Insert a tree at specified location in the ensemble
Parameters: |
---|
Example
builder = ModelBuilder(num_feature=4227)
tree = ... # build tree somehow
builder.insert(tree, 0) # insert tree at index 0
treelite.
Annotator
(path=None)¶Branch annotator class: annotate branches in a given model using frequency patterns in the training data
Parameters: | path (str , optional) – if given, the predictor will load branch frequency information
from the path |
---|
annotate_branch
(model, dmat, nthread=None, verbose=False)¶Annotate branches in a given model using frequency patterns in the training data. Each node gets the count of the instances that belong to it. Any prior annotation information stored in the annotator will be replaced with the new annotation returned by this method.
Parameters: |
|
---|
Create shared library.
Parameters: |
|
---|---|
Returns: | libpath – absolute path of created shared library |
Return type: |
Example
The following command uses Visual C++ toolchain to generate
./my/model/model.dll
:
model.compile(dirpath='./my/model', params={}, verbose=True)
create_shared(toolchain='msvc', dirpath='./my/model', verbose=True)
Later, the shared library can be referred to by its directory name:
predictor = Predictor(libpath='./my/model', verbose=True)
# looks for ./my/model/model.dll
Alternatively, one may specify the library down to its file name:
predictor = Predictor(libpath='./my/model/model.dll', verbose=True)
treelite.
save_runtime_package
(destdir, include_binary=False)¶Save a copy of the (zipped) runtime package, containing all glue code necessary to deploy compiled models into the wild
Parameters: |
|
---|
treelite.
generate_makefile
(dirpath, platform, toolchain, options=None)¶Generate a Makefile for a given directory of headers and sources. The resulting Makefile will be stored in the directory. This function is useful for deploying a model on a different machine.
Parameters: |
|
---|
treelite.gallery.sklearn.
import_model
(sklearn_model)¶Load a tree ensemble model from a scikit-learn model object
Parameters: | sklearn_model (object of type RandomForestRegressor / RandomForestClassifier / GradientBoostingRegressor / GradientBoostingClassifier ) – Python handle to scikit-learn model |
---|---|
Returns: | model – loaded model |
Return type: | Model object |
Example
import sklearn.datasets
import sklearn.ensemble
X, y = sklearn.datasets.load_boston(return_X_y=True)
clf = sklearn.ensemble.RandomForestRegressor(n_estimators=10)
clf.fit(X, y)
import treelite.gallery.sklearn
model = treelite.gallery.sklearn.import_model(clf)
Runtime API of treelite Python package.
Runtime API provides the minimum necessary tools to deploy tree prediction modules in the wild.
treelite.runtime.
Predictor
(libpath, nthread=None, verbose=False, include_master_thread=True)¶Predictor class: loader for compiled shared libraries
Parameters: |
|
---|
predict
(batch, verbose=False, pred_margin=False)¶Make prediction using a batch of data rows (synchronously). This will internally split workload among worker threads.
Parameters: |
---|
treelite.runtime.
Batch
¶Batch of rows to be used for prediction
from_csr
(csr, rbegin=None, rend=None)¶Get a sparse batch from a subset of rows in a CSR (Compressed Sparse Row)
matrix. The subset is given by the range [rbegin, rend)
.
Parameters: |
|
---|---|
Returns: | sparse_batch – a sparse batch consisting of rows |
Return type: |
from_npy2d
(mat, rbegin=0, rend=None, missing=None)¶Get a dense batch from a 2D numpy matrix.
If mat
does not have order='C'
(also known as row-major) or is not
contiguous, a temporary copy will be made.
If mat
does not have dtype=numpy.float32
, a temporary copy will be
made also.
Thus, as many as two temporary copies of data can be made. One should set
input layout and type judiciously to conserve memory.
Parameters: |
|
---|---|
Returns: | dense_batch – a dense batch consisting of rows |
Return type: |
Treelite exposes a set of C functions to enable interfacing with a variety of languages. This page will be most useful for:
We recommend the Python API for everyday uses.
Note
Use of C and C++ in treelite
Core logic of treelite are written in C++ to take advantage of higher abstractions. We provide C only interface here, as many more programming languages bind with C than with C++. See this page for more details.
Contents
Use the following functions to load and manipulate data from a variety of sources.
TreeliteDMatrixCreateFromFile
(const char *path, const char *format, int nthread, int verbose, DMatrixHandle *out)¶create DMatrix from a file
path
: file path format
: file format nthread
: number of threads to use verbose
: whether to produce extra messages out
: the created DMatrix
TreeliteDMatrixCreateFromCSR
(const float *data, const unsigned *col_ind, const size_t *row_ptr, size_t num_row, size_t num_col, DMatrixHandle *out)¶create DMatrix from a (in-memory) CSR matrix
data
: feature values col_ind
: feature indices row_ptr
: pointer to row headers num_row
: number of rows num_col
: number of columns out
: the created DMatrix
TreeliteDMatrixCreateFromMat
(const float *data, size_t num_row, size_t num_col, float missing_value, DMatrixHandle *out)¶create DMatrix from a (in-memory) dense matrix
data
: feature values num_row
: number of rows num_col
: number of columns missing_value
: value to represent missing value out
: the created DMatrix
TreeliteDMatrixGetDimension
(DMatrixHandle handle, size_t *out_num_row, size_t *out_num_col, size_t *out_nelem)¶get dimensions of a DMatrix
handle
: handle to DMatrix out_num_row
: used to set number of rows out_num_col
: used to set number of columns out_nelem
: used to set number of nonzero entries
TreeliteDMatrixGetPreview
(DMatrixHandle handle, const char **out_preview)¶produce a human-readable preview of a DMatrix Will print first and last 25 non-zero entries, along with their locations
handle
: handle to DMatrix out_preview
: used to save the address of the string literal
TreeliteDMatrixGetArrays
(DMatrixHandle handle, const float **out_data, const uint32_t **out_col_ind, const size_t **out_row_ptr)¶extract three arrays (data, col_ind, row_ptr) that define a DMatrix.
handle
: handle to DMatrix out_data
: used to save pointer to array containing feature values out_col_ind
: used to save pointer to array containing feature indices out_row_ptr
: used to save pointer to array containing pointers to row headers
TreeliteDMatrixFree
(DMatrixHandle handle)¶delete DMatrix from memory
handle
: handle to DMatrix Use the following functions to annotate branches in decision trees.
TreeliteAnnotateBranch
(ModelHandle model, DMatrixHandle dmat, int nthread, int verbose, AnnotationHandle *out)¶annotate branches in a given model using frequency patterns in the training data.
model
: model to annotate dmat
: training data matrix nthread
: number of threads to use verbose
: whether to produce extra messages out
: used to save handle for the created annotation
TreeliteAnnotationLoad
(const char *path, AnnotationHandle *out)¶load branch annotation from a JSON file
path
: path to JSON file out
: used to save handle for the loaded annotation
TreeliteAnnotationSave
(AnnotationHandle handle, const char *path)¶save branch annotation to a JSON file
handle
: annotation to save path
: path to JSON file
TreeliteAnnotationFree
(AnnotationHandle handle)¶delete branch annotation from memory
handle
: annotation to remove Use the following functions to produce optimize prediction subroutine (in C) from a given decision tree ensemble.
TreeliteCompilerCreate
(const char *name, CompilerHandle *out)¶create a compiler with a given name
name
: name of compiler out
: created compiler
TreeliteCompilerSetParam
(CompilerHandle handle, const char *name, const char *value)¶set a parameter for a compiler
handle
: compiler name
: name of parameter value
: value of parameter
TreeliteCompilerGenerateCode
(CompilerHandle compiler, ModelHandle model, int verbose, const char *dirpath)¶generate prediction code from a tree ensemble model. The code will be C99 compliant. One header file (.h) will be generated, along with one or more source files (.c).
Usage example:
TreeliteCompilerGenerateCode(compiler, model, 1, "./my/model");
// files to generate: ./my/model/header.h, ./my/model/main.c
// if parallel compilation is enabled:
// ./my/model/header.h, ./my/model/main.c, ./my/model/tu0.c,
// ./my/model/tu1.c, and so forth
compiler
: handle for compiler model
: handle for tree ensemble model verbose
: whether to produce extra messages dirpath
: directory to store header and source files
TreeliteCompilerFree
(CompilerHandle handle)¶delete compiler from memory
handle
: compiler to remove Use the following functions to load decision tree ensemble models from a file. Treelite supports multiple model file formats.
TreeliteLoadLightGBMModel
(const char *filename, ModelHandle *out)¶load a model file generated by LightGBM (Microsoft/LightGBM). The model file must contain a decision tree ensemble.
filename
: name of model file out
: loaded model
TreeliteLoadXGBoostModel
(const char *filename, ModelHandle *out)¶load a model file generated by XGBoost (dmlc/xgboost). The model file must contain a decision tree ensemble.
filename
: name of model file out
: loaded model
TreeliteLoadXGBoostModelFromMemoryBuffer
(const void *buf, size_t len, ModelHandle *out)¶load an XGBoost model from a memory buffer.
buf
: memory buffer len
: size of memory buffer out
: loaded model
TreeliteLoadProtobufModel
(const char *filename, ModelHandle *out)¶load a model in Protocol Buffers format. Protocol Buffers (google/protobuf) is a language- and platform-neutral mechanism for serializing structured data. See tree.proto for format spec.
filename
: name of model file out
: loaded model
TreeliteExportXGBoostModel
(const char *filename, ModelHandle model, const char *name_obj)¶(EXPERIMENTAL FEATURE) export a model in XGBoost format. The exported model can be read by XGBoost (dmlc/xgboost).
filename
: name of model file model
: model to export name_obj
: name of objective function
TreeliteFreeModel
(ModelHandle handle)¶delete model from memory
handle
: model to remove Use the following functions to incrementally build decisio n tree ensemble models.
TreeliteCreateTreeBuilder
(TreeBuilderHandle *out)¶Create a new tree builder.
out
: newly created tree builder
TreeliteDeleteTreeBuilder
(TreeBuilderHandle handle)¶Delete a tree builder from memory.
handle
: tree builder to remove
TreeliteTreeBuilderCreateNode
(TreeBuilderHandle handle, int node_key)¶Create an empty node within a tree.
handle
: tree builder node_key
: unique integer key to identify the new node
TreeliteTreeBuilderDeleteNode
(TreeBuilderHandle handle, int node_key)¶Remove a node from a tree.
handle
: tree builder node_key
: unique integer key to identify the node to be removed
TreeliteTreeBuilderSetRootNode
(TreeBuilderHandle handle, int node_key)¶Set a node as the root of a tree.
handle
: tree builder node_key
: unique integer key to identify the root node
TreeliteTreeBuilderSetNumericalTestNode
(TreeBuilderHandle handle, int node_key, unsigned feature_id, const char *opname, float threshold, int default_left, int left_child_key, int right_child_key)¶Turn an empty node into a test node with numerical split. The test is in the form [feature value] OP [threshold]. Depending on the result of the test, either left or right child would be taken.
handle
: tree builder node_key
: unique integer key to identify the node being modified; this node needs to be empty feature_id
: id of feature opname
: binary operator to use in the test threshold
: threshold value default_left
: default direction for missing values left_child_key
: unique integer key to identify the left child node right_child_key
: unique integer key to identify the right child node
TreeliteTreeBuilderSetCategoricalTestNode
(TreeBuilderHandle handle, int node_key, unsigned feature_id, const unsigned int *left_categories, size_t left_categories_len, int default_left, int left_child_key, int right_child_key)¶Turn an empty node into a test node with categorical split. A list defines all categories that would be classified as the left side. Categories are integers ranging from 0 to (n-1), where n is the number of categories in that particular feature. Let’s assume n <= 64.
handle
: tree builder node_key
: unique integer key to identify the node being modified; this node needs to be empty feature_id
: id of feature left_categories
: list of categories belonging to the left child left_categories_len
: length of left_cateogries default_left
: default direction for missing values left_child_key
: unique integer key to identify the left child node right_child_key
: unique integer key to identify the right child node
TreeliteTreeBuilderSetLeafNode
(TreeBuilderHandle handle, int node_key, float leaf_value)¶Turn an empty node into a leaf node.
handle
: tree builder node_key
: unique integer key to identify the node being modified; this node needs to be empty leaf_value
: leaf value (weight) of the leaf node
TreeliteTreeBuilderSetLeafVectorNode
(TreeBuilderHandle handle, int node_key, const float *leaf_vector, size_t leaf_vector_len)¶Turn an empty node into a leaf vector node The leaf vector (collection of multiple leaf weights per leaf node) is useful for multi-class random forest classifier.
handle
: tree builder node_key
: unique integer key to identify the node being modified; this node needs to be empty leaf_vector
: leaf vector of the leaf node leaf_vector_len
: length of leaf_vector
TreeliteCreateModelBuilder
(int num_feature, int num_output_group, int random_forest_flag, ModelBuilderHandle *out)¶Create a new model builder.
num_feature
: number of features used in model being built. We assume that all feature indices are between 0 and (num_feature - 1). num_output_group
: number of output groups. Set to 1 for binary classification and regression; >1 for multiclass classification random_forest_flag
: whether the model is a random forest. Set to 0 if the model is gradient boosted trees. Any nonzero value shall indicate that the model is a random forest. out
: newly created model builder
TreeliteModelBuilderSetModelParam
(ModelBuilderHandle handle, const char *name, const char *value)¶Set a model parameter.
handle
: model builder name
: name of parameter value
: value of parameter
TreeliteDeleteModelBuilder
(ModelBuilderHandle handle)¶Delete a model builder from memory.
handle
: model builder to remove
TreeliteModelBuilderInsertTree
(ModelBuilderHandle handle, TreeBuilderHandle tree_builder, int index)¶Insert a tree at specified location.
handle
: model builder tree_builder
: builder for the tree to be inserted. The tree must not be part of any other existing tree ensemble. Note: The tree_builder argument will become unusuable after the tree insertion. Should you want to modify the tree afterwards, use GetTree(*) method to get a fresh handle to the tree. index
: index of the element before which to insert the tree; use -1 to insert at the end
TreeliteModelBuilderGetTree
(ModelBuilderHandle handle, int index, TreeBuilderHandle *out)¶Get a reference to a tree in the ensemble.
handle
: model builder index
: index of the tree in the ensemble out
: used to save reference to the tree
TreeliteModelBuilderDeleteTree
(ModelBuilderHandle handle, int index)¶Remove a tree from the ensemble.
handle
: model builder index
: index of the tree that would be removed
TreeliteModelBuilderCommitModel
(ModelBuilderHandle handle, ModelHandle *out)¶finalize the model and produce the in-memory representation
handle
: model builder out
: used to save handle to in-memory representation of the finished model Use the following functions to load compiled prediction subroutines from shared libraries and to make predictions.
TreeliteAssembleSparseBatch
(const float *data, const uint32_t *col_ind, const size_t *row_ptr, size_t num_row, size_t num_col, CSRBatchHandle *out)¶assemble a sparse batch
data
: feature values col_ind
: feature indices row_ptr
: pointer to row headers num_row
: number of data rows in the batch num_col
: number of columns (features) in the batch out
: handle to sparse batch
TreeliteDeleteSparseBatch
(CSRBatchHandle handle)¶delete a sparse batch from memory
handle
: sparse batch
TreeliteAssembleDenseBatch
(const float *data, float missing_value, size_t num_row, size_t num_col, DenseBatchHandle *out)¶assemble a dense batch
data
: feature values missing_value
: value to represent the missing value num_row
: number of data rows in the batch num_col
: number of columns (features) in the batch out
: handle to sparse batch
TreeliteDeleteDenseBatch
(DenseBatchHandle handle)¶delete a dense batch from memory
handle
: dense batch
TreeliteBatchGetDimension
(void *handle, int batch_sparse, size_t *out_num_row, size_t *out_num_col)¶get dimensions of a batch
handle
: a batch of rows (must be of type SparseBatch or DenseBatch) batch_sparse
: whether the batch is sparse (true) or dense (false) out_num_row
: used to set number of rows out_num_col
: used to set number of columns
TreelitePredictorLoad
(const char *library_path, int num_worker_thread, int include_master_thread, PredictorHandle *out)¶load prediction code into memory. This function assumes that the prediction code has been already compiled into a dynamic shared library object (.so/.dll/.dylib).
library_path
: path to library object file containing prediction code num_worker_thread
: number of worker threads (-1 to use max number) include_master_thread
: whether to assign workload to the master thread. If not, only workers threads will be assigned work. out
: handle to predictor
TreelitePredictorPredictBatch
(PredictorHandle handle, void *batch, int batch_sparse, int verbose, int pred_margin, float *out_result, size_t *out_result_size)¶Make predictions on a batch of data rows (synchronously). This function internally divides the workload among all worker threads.
handle
: predictor batch
: a batch of rows (must be of type SparseBatch or DenseBatch) batch_sparse
: whether batch is sparse (1) or dense (0) verbose
: whether to produce extra messages pred_margin
: whether to produce raw margin scores instead of transformed probabilities out_result
: resulting output vector; use TreelitePredictorQueryResultSize() to allocate sufficient space out_result_size
: used to save length of the output vector, which is guaranteed to be less than or equal to TreelitePredictorQueryResultSize()
TreelitePredictorQueryResultSize
(PredictorHandle handle, void *batch, int batch_sparse, size_t *out)¶Given a batch of data rows, query the necessary size of array to hold predictions for all data points.
handle
: predictor batch
: a batch of rows (must be of type SparseBatch or DenseBatch) batch_sparse
: whether batch is sparse (1) or dense (0) out
: used to store the length of prediction array
TreelitePredictorQueryNumOutputGroup
(PredictorHandle handle, size_t *out)¶Get the number of output groups in the loaded model The number is 1 for most tasks; it is greater than 1 for multiclass classifcation.
handle
: predictor out
: length of prediction array
TreelitePredictorFree
(PredictorHandle handle)¶delete predictor from memory
handle
: predictor to remove Treelite uses C++ classes to define its internal data structures. In order to
pass C++ objects to C functions, opaque handles are used. Opaque handles
are void*
pointers that store raw memory addresses.
DMatrixHandle
¶handle to a data matrix
ModelHandle
¶handle to a decision tree ensemble model
TreeBuilderHandle
¶handle to tree builder class
ModelBuilderHandle
¶handle to ensemble builder class
AnnotationHandle
¶handle to branch annotation data
CompilerHandle
¶handle to compiler class
PredictorHandle
¶handle to predictor class
CSRBatchHandle
¶handle to batch of sparse data rows
DenseBatchHandle
¶handle to batch of dense data rows
Compiler parameters influence the way the prediction subroutine is generated from a tree ensemble model.
annotate_in
¶name of model annotation file. Use the class treelite.Annotator
to generate this file.
quantize
¶whether to quantize threshold points (0: no, >0: yes)
parallel_comp
¶option to enable parallel compilation; if set to nonzero, the trees will be evely distributed into [parallel_comp]
files. Set this option to improve compilation time and reduce memory consumption during compilation.
verbose
¶if >0, produce extra messages
Model parameters are used by ModelBuilder
class to
clarify certain behaviors of a tree ensemble model.
pred_transform
¶name of prediction transform function
This parameter specifies how to transform raw margin values into final predictions. By default, this is set to 'identity'
, which means no transformation.
For the multi-class classification task, pred_transfrom
must be one of the following values:
- identity_multiclass
do not transform. The output will be a matrix with dimensions
[number of data points] * [number of classes] that contains the margin score
for every (data point, class) pair.
- max_index
compute the most probable class for each data point and output the class
index. The output will be a vector of length [number of data points] that
contains the most likely class of each data point.
- softmax
use the softmax function to transform a multi-dimensional vector into a
proper probability distribution. The output will be a matrix with dimensions
[number of data points] * [number of classes] that contains the predicted
probability of each data point belonging to each class.
- multiclass_ova
apply the sigmoid function element-wise to the margin matrix. The output will
be a matrix with dimensions [number of data points] * [number of classes].
pred_transfrom
must be one of the following values: - identity
do not transform. The output will be a vector of length
[number of data points] that contains the margin score for every data point.
- sigmoid
apply the sigmoid function element-wise to the margin vector. The output
will be a vector of length [number of data points] that contains the
probability of each data point belonging to the positive class.
- exponential
apply the exponential function (exp) element-wise to the margin vector. The
output will be a vector of length [number of data points].
- logarithm_one_plus_exp
apply the function f(x) = log(1 + exp(x)) element-wise to the margin vector.
The output will be a vector of length [number of data points].
sigmoid_alpha
¶scaling parameter for sigmoid function sigmoid(x) = 1 / (1 + exp(-alpha * x))
This parameter is used only when pred_transform
is set to 'sigmoid'
. It must be strictly positive; if unspecified, it is set to 1.0.
global_bias
¶global bias of the model
Predicted margin scores of all instances will be adjusted by the global bias. If unspecified, the bias is set to zero.