Hyperparameter tuning

This guide walks you through the step by step process of tuning hyperparameters in the MathFi.ai platform to optimize training and gradually obtain better model performance. You can perform tuning interactively using the dashboard or programmatically via the REST API.

​

What are the hyperparameters?

There are only 3 training hyperparameters within the MathFi.ai platform.

​

Hyperparameter tuning video

You can continue reading this guide or watch a full example walkthrough of hyperparameter tuning in this video:

​

Initial baseline training

• Number of Buckets: 20
• Scaling Factor: 19
• Performance Threshold: 0.8

See the Training Guide to run this initial setup. The result should be an initial champion model.

​

Step-by-Step tuning process

​

1. Start by increasing the Performance Threshold

Following up with the initial training from the previous step:

• If no model achieves > 0.8 accuracy, the training will stop with status Not Completed
• If this is the case, inspect which algorithm came closest
• For example, if one algorithm reached 0.78:
  • Lower Performance Threshold to 0.78
  • Rerun training to attempt getting a new Champion Model with performance 0.78

​

2. Gradually increase the threshold

If MathFi.ai succeeds with the newly chosen threshold ( 0.8, 0.78, …), continue increasing it in 0.05 increments:

• Set threshold to 0.85, then 0.90, then 0.95

With each increase:

• Monitor if a new Champion Model is selected (training completes successfully with the chosen threshold)
• Watch for overfitting — train and test performance should stay close

​

3. Handle failure at high thresholds

If you reach a threshold like 0.95 (or way lower) and training fails (e.g., Timeout or Not Completed), there are mainly 2 options to continue:

• Accept the lower accuracy or
• Move to tune other parameters

Option A: Accept a lower accuracy

If 0.93 is acceptable (as the max seen in the Performance column):

• Lower Performance Threshold to 0.93
• Rerun training

If this value wasn’t acceptable, move to tune other parameters and keep the original, desired threshold.

Option B: Tune other parameters

• Keep Performance Threshold at your desired outcome (e.g. 0.95)
• Adjust Number of Buckets or Scaling Factor
  • e.g., Lower Number of Buckets from 20 → 10
  • For this, create a new dataset version with a different name (e.g., Account Fraud 3)

• Rerun training with this new dataset version, maintaining the desired threshold

• If a training reaches a very close value (e.g. 0.94), update threshold to that value and rerun

​

4. Repeat as needed

Continue this iterative process to tune performance:

• Adjust one hyperparameter at a time
• Log your results
• Stop when you reach desired accuracy or a maximum once enough combinations tried

​

Sample tuning iteration

​

Automating the process

All of the previous steps can be performed via the REST API:

• Upload CSV, create a dataset
• Run training
• Check results
• Re-run training with adjusted parameters or create another dataset version with new values
• Eventually, download predictions

​

Summary

Need help tuning a specific dataset? Send us a support request. See real tuning progressions in the use cases: Credit Card Approval and Self-checkout Fraud both show multi-step iterations that push performance above 95%.