Question

If we only have a small number of observations, k-fold cross-validation provides a better estimate of the generalization error than the validation set method.

Answer 1

True,

If the number of observations is small. Then partitioning the data into train, test and validation set can result in losing a lot of information. Train set can be a good representative of data if there are enough number of observations. We know that most of the ML algorithms need a large amount of data to be trained. Similarly, the validation set will be very small for small data. Using k-fold cross validation we can overcome that. One of the main reasons for using K-fold cross-validation instead of using conventional validation is that there is not sufficient data present to partition it into training and test sets without sacrificing significant modelling or testing capability. A fair way to properly estimate model performance is to use K-fold cross-validation in these cases.