1. Executive Summary

The objective was to build a supervised machine-learning model that predicts automobile price from historical vehicle data. The project followed a professional model-development workflow: data inspection, missing-value treatment, feature/target separation, categorical encoding, train/test separation, baseline regression, feature analysis, polynomial regression, complexity analysis, Ridge regularization, alpha tuning, cross-validation, final model selection, untouched test evaluation, and a new-car prediction.

The most important learning was not simply the final score. The project demonstrated how model selection should be separated from final testing, why model complexity can cause overfitting, why regularization controls coefficient size, why scaling matters for Ridge, and why a single validation split can give an overly optimistic impression.

The selected model at the end of the completed cross-validation workflow was a degree-2 polynomial Ridge model with alpha = 100. Its mean CV RMSE was approximately $2,930.17. On the untouched 41-car test set, it produced MAE $3,702.07, RMSE $5,493.68, and R² 0.7533.

A new example car was then passed through the final pipeline and received a predicted price of $11,292.68. This is a model estimate, not a guaranteed market value.

2. Business / Real-World Problem

Given historical automobile specifications and characteristics, estimate the market price of a vehicle. In a real application, such a model could support a used-car marketplace, valuation tool, inventory pricing workflow, or internal analytics system.

The target variable is price. The challenge is that price is affected by many interacting factors, while the dataset is relatively small. A useful model therefore needs to balance predictive power against overfitting.

3. Dataset Understanding

The cleaned dataset contained 201 rows and 26 columns: 25 candidate input features plus the price target.

The dataset contained both numerical and categorical variables.

Before cleaning, missing values were present. The largest missing-value count was 41 for normalized-losses. Price had 4 missing values; stroke and bore had 4 each; peak-rpm had 2; num-of-doors had 2; and horsepower had 2.

After missing-value treatment, every column had zero missing values and the dataset contained 201 × 26.

4. Cleaning and Feature Preparation

4.1 Missing Values

Missing values were handled before modeling. The final missing-value check showed zero missing values in all columns, including price.

4.2 Separate Features and Target

The target was price, while the remaining columns formed the input matrix X.

4.3 Categorical Encoding

Categorical variables cannot be directly consumed by ordinary scikit-learn linear regression in text form, so one-hot encoding was used.

The increase from 25 to 64 columns is expected because a single categorical column can become multiple binary indicator columns.

5. Train/Test Split

The data was split into 160 development/training observations and 41 test observations.

6. Baseline Linear Regression

A baseline Linear Regression model was trained first. A baseline establishes a reference point so that later complexity has to demonstrate a real benefit.

Example predictions included actual $8,249 versus predicted $7,307.84, actual $41,315 versus predicted $28,075.75, and actual $35,056 versus predicted $43,457.28.

7. Feature Analysis and Correlation

We examined numerical correlations with price as an exploratory analysis. The purpose was to understand which numerical variables had strong linear relationships with price—not to automatically decide which features must enter the final model.

8. Why We Introduced a Five-Feature Polynomial Experiment

For the educational polynomial/regularization experiment, we deliberately restricted the inputs to five meaningful numerical features:

This was a controlled experiment, not a claim that these five were the best possible final features.

The reason was practical: polynomial expansion grows rapidly. With five inputs, degree 4 already generated 126 polynomial features. That made the effects of complexity and regularization visible without creating an unnecessarily enormous feature space.

This distinction became important later: a model built on five features cannot be used as evidence that the dataset lacks useful information when many other features are available.

9. Why Validation Became Important

During model development, we corrected an important methodological issue by separating the 160 development observations into training and validation portions for the polynomial complexity experiment. The 41-car test set remained untouched.

This avoids repeatedly looking at the test set and selecting whichever model happens to perform best on it.

10. Polynomial Regression and Complexity

Polynomial regression was introduced to capture nonlinear relationships that a straight line cannot represent. Multiple degrees were tested rather than assuming a particular degree was best.

In the refined complexity experiment, degree 2 produced the best validation RMSE among the tested unregularized polynomial degrees.

The key lesson was that higher degree is not automatically better. Increasing degree increases flexibility and creates more coefficients. If validation error rises after a certain degree, there is no reason to choose the higher degree simply because it is more complex.

11. Ridge Regression — Why Regularization?

Ridge Regression adds an L2 penalty to the ordinary loss. Conceptually:

The penalty discourages unnecessarily large coefficients. Shrinking coefficients can reduce sensitivity to small changes in the training data and can improve generalization when an unregularized model is too flexible.

Ridge is not magic and is not automatically required. If a simpler model performs better on validation/CV, the simpler model should be selected.

12. First Ridge Experiment — An Important Failure

We applied Ridge to a high-complexity degree-4 polynomial model. The initial result was:

This was severe overfitting: training error was extremely small while validation error was enormous.

Testing several alpha values without proper scaling produced a best alpha of 10, but the best validation RMSE was still approximately $20,591.56.

This was an important diagnostic moment. Ridge had improved the result relative to no regularization, but the setup was still poor. We did not conclude that Ridge itself was useless.

13. Scaling + Ridge

We then identified an important implementation issue. Ridge penalizes coefficients, so features should be on comparable scales. Polynomial terms also have very different magnitudes.

With degree 4, scaling, and Ridge alpha=1:

This demonstrated why preprocessing is part of the model rather than an optional cosmetic step. The same Ridge idea behaved very differently once feature scale was handled correctly.

14. Tuning Ridge Alpha

We tested multiple alpha values. The best result in that experiment was alpha = 0.1 with validation RMSE approximately $2,554.53.

This was better than the degree-2 unregularized validation result of approximately $2,919.27 on that particular validation split. At this point, degree 4 + Ridge looked promising—but we had only evaluated one validation split, so we did not declare victory yet.

15. Five-Fold Cross-Validation

To reduce dependence on one arbitrary validation split, we performed 5-fold cross-validation on the 160 development observations.

The large variation between folds showed that model performance was not equally stable across subsets. This is exactly why a single validation score should be interpreted cautiously.

16. Joint Selection of Polynomial Degree and Alpha

We evaluated multiple polynomial degrees and Ridge alpha values using the same 5-fold cross-validation framework.

17. Final Model Training

After model selection, the chosen pipeline was retrained on all 160 development observations. The test set was still not used for fitting or selection.

This is the correct point to use all available development data: the validation/CV stage has already served its model-selection purpose.

18. Final Test-Set Evaluation

The final model was then evaluated once on the 41 completely untouched test cars:

This was an important reality check. The mean CV RMSE was about $2,930, but the final test RMSE was about $5,494.

We should not go back and tune the model based on the test result. Doing so would turn the test set into another validation set and weaken the credibility of the final evaluation.

19. Diagnosing the Large Test Error

Rather than simply accepting the test error, we investigated whether the problem was model overfitting, data distribution, or insufficient features.

The test set had substantially higher price variability. The largest test errors were concentrated among expensive vehicles.

The model systematically underpredicted several high-priced cars. Because RMSE squares errors, a handful of very large mistakes can raise RMSE substantially.

The evidence did not justify saying simply that “Ridge failed because of overfitting.” Training and validation behavior in the properly scaled Ridge experiment were much closer than the severe overfitting seen in the initial unscaled experiment.

20. A Critical Feature-Selection Lesson

The five-feature polynomial/Ridge experiment was an educational controlled experiment. It was not a proof that only five variables should be used in a final car-price model.

The original dataset contained many potentially informative variables:

The expensive-car underprediction therefore led to a stronger conclusion:

21. Why the $11,292.68 New-Car Prediction Is Still Useful — But Limited

The final pipeline was used to predict an example new car. The model returned:

The $11,292.68 value is the model’s best single numerical estimate given the inputs and learned relationships. However, it should not be presented as an exact market value.

The final test RMSE demonstrates that the model can make substantially larger errors, particularly for some expensive vehicles. A production valuation system would need stronger predictive performance and/or uncertainty reporting before the output should be treated as highly reliable.

22. Final Conclusions

1. The project successfully implemented a complete supervised-learning workflow from raw data to a new prediction.
2. Missing-value handling and categorical encoding were necessary preprocessing steps.
3. Correlation was useful for understanding relationships but was not treated as proof of causation or as automatic feature selection.
4. Polynomial degree controls model flexibility. Higher degree is not automatically better; degree 2 was favored over higher degrees in the relevant experiments.
5. Ridge regularization shrinks coefficients and can stabilize a complex model, but it must be tuned and used with appropriate scaling.
6. Cross-validation is more reliable for model selection than trusting one arbitrary validation split.
7. The selected CV model was degree 2 + Ridge alpha 100, with mean CV RMSE about $2,930.17.
8. The untouched test set produced MAE $3,702.07, RMSE $5,493.68, and R² 0.7533.
9. The test errors were concentrated among several expensive vehicles, and the test set had greater price variability than the training set.
10. We should not immediately conclude that more data is required.
11. A major next experiment is to use the full set of useful numerical and categorical features with leakage-safe preprocessing and cross-validation.
12. If a properly engineered full-feature model still performs poorly across representative splits, then there would be stronger evidence that dataset size, coverage, noise, or missing explanatory variables are limiting performance.

23. Current Project Status and Next Experiment

At the point this case study was prepared, the next diagnostic experiment had been defined but its result had not yet been produced. The plan is to use all 25 original input features rather than the five-feature educational subset.

This experiment is intentionally separate from the polynomial experiment. First, we test whether simply restoring useful information improves prediction. Only after measuring that result should we decide whether additional nonlinear features, other algorithms, or more data are necessary.

24. Reproducibility Checklist

• Keep the 41-car test set untouched during model selection.
• Fit encoders and scalers only on training folds; use Pipeline/ColumnTransformer to prevent leakage.
• Use cross-validation to compare candidate models and hyperparameters.
• Report MAE, RMSE, and R² together; do not rely on one metric alone.
• Inspect large residuals rather than looking only at aggregate scores.
• Compare price distributions across development and test data.
• Do not claim that more data is necessary until feature coverage and model specification have been tested.
• After final model selection, train on all development data and evaluate the test set once.

25. Key Machine-Learning Lessons

Coefficient

In linear and Ridge regression, a coefficient is the model parameter multiplying a feature. For a simple one-feature linear model, it is the slope. In a multi-feature model, each coefficient represents the model’s estimated change associated with that feature while the other included features are held constant, subject to the model’s assumptions.

Overfitting

A model can fit training data extremely well while performing poorly on unseen data. Regularization can reduce this by discouraging large coefficients, but the amount of regularization must be selected from development data.

Model Selection vs. Final Evaluation

Validation/CV is for choosing. The test set is for the final reality check.

Prediction vs. Guarantee

A predicted value is a point estimate. Its usefulness depends on how accurately the model generalizes to new observations.

Scientific Workflow

When performance is poor, diagnose the source of the problem before changing the model. A worse test result can reveal distribution differences, missing features, model limitations, or data limitations.

Appendix A — Main Code Patterns Used

Appendix B — Final Result Snapshot