The following figure shows the prediction throughput of treelite and XGBoost, measured with various batch sizes.

(Get this plot in SVG, PNG, High-resolution PNG)

System configuration. One AWS EC2 instance of type c5.18xlarge was used. It consists of the following components:

  • CPU: 72 virtual cores, 64-bit
  • Memory: 144 GB
  • Storage: Elastic Block Storage (EBS)
  • Operating System: Ubuntu 14.04.5 LTS

Datasets. Three datasets were used.

Methods. For each datasets, we trained a 1600-tree ensemble using XGBoost. Then we made predictions on batches of various sizes that were sampled randomly from the training data. After running predictions using treelite and XGBoost (latter with xgboost.Booster.predict()), we measured throughput as the number of lines predicted per second.

Download the benchmark script:

Actual measurements. You may download the exact measurements using the following links: