Support Vector Machine Settings for Regression

The settings that you can specify for the Support Vector Machine (SVM) algorithm depend on the kernel that you select.

The default settings are designed so that they should work for most cases.

Click Restore to restore the default values.

When you are done specifying algorithm settings, click OK to continue.

Kernel Function

SVM supports two kernel functions: Linear (the default) and Gaussian. For more information, see SVM Kernel Functions.

Algorithm Settings for Linear Kernel or System Determined Kernel

If you specify the Linear Kernel or if you let the system determine the kernel, you can change the following settings:

Algorithm Settings for Gaussian Kernel

If you specify the Gaussian kernel, you can change the following settings:

Active Learning

SVM models grow as the size of the training data set increases. This property limits SVM models to small and medium size build data sets (less than 100,000 cases). Active learning provides a way to deal with large build data sets.

Active Learning is a methodology optimizes the selection of a subset of the support vectors that maintain accuracy while enhancing the speed of the model.

Active learning increases performance for a linear kernel. Active learning both increases performance and reduces the size of the Gaussian kernel; this is an important consideration if memory and temporary disk space are issues.

Active learning forces the SVM algorithm to restrict learning to the most informative examples and not to attempt to use the entire body of data. In most cases, the resulting models have predictive accuracy comparable to that of the standard (exact) SVM model.

In most cases, you should not disable this setting.

Active learning is on by default. It can be turned off by answering No to the Do you Want Active Learning?.

Cache Size for Gaussian Kernel

If you select the Gaussian kernel, you can specify a cache size for the size of the cache used for storing computed kernels during the build operation. The default size is 50 megabytes.

The most expensive operation in building a Gaussian SVM model is the computation of kernels. The general approach taken to build is to converge within a chunk of data at a time, then to test for violators outside of the chunk. Build is complete when there are no more violators within tolerance. The size of the chunk is chosen such that the associated kernels can be maintained in memory in a "Kernel Cache". The larger the chunk size, the better the chunk represents the population of training data and the fewer number of times new chunks will need to be created. Generally, larger caches imply faster builds.

Complexity Factor

You specify the complexity factor for an SVM model by clicking the radio button next to Yes in answer to the question Do you want to specify a complexity factor for predictor loss?

The complexity factor determines the trade-off between minimizing model error on the training data and minimizing model complexity. Its responsibility is to avoid over-fit (an over-complex model fitting noise in the training data) and under-fit (a model that is too simple).

A very large value of the complexity factor places an extreme penalty on errors, so that SVM seeks a perfect separation of target classes. A small value for the complexity factor places a low penalty on errors and high constraints on the model parameters, which can lead to under-fit.

If the histogram of the target attribute is skewed to the left or to the right, try increasing complexity.

The default is to specify no complexity factor, in which case the system calculates a complexity factor. If you do specify a complexity factor, specify a positive number.

Epsilon Value

You specify the epsilon value for an SVM model by clicking the radio button next to Yes in answer to the question Do you want to specify an epsilon value?. The epsilon value must be either greater than 0 or undefined.

SVM makes a distinction between small errors and large errors; the difference is defined by the epsilon value. The algorithm will calculate and optimize an epsilon value internally, or you can supply a value. If the number of support vectors defined by the model is very large, try a larger epsilon value; if there are very high cardinality categorical attributes, try decreasing epsilon.

The default is to specify no epsilon value, in which case the algorithm calculates an epsilon value.

Standard Deviation for Gaussian Kernel

If you select the Gaussian kernel, you can specify the standard deviation of the Gaussian kernel. This value must be positive. The default is to not set the standard deviation.

Tolerance Value

Tolerance value is the maximum size of a violation of convergence criteria such that the model is considered to have converged. The default value is 0.001. Larger values imply faster building but less accurate models.