Spectroscopy Analysis with Machine Learning Regression for the Quantification of Carbon and Nitrogen Contents in Inceptisol and Oxisol Soil Types: Comparing Different Preprocessing and Validation methods as well as Feature Importance
Pith reviewed 2026-07-02 15:53 UTC · model grok-4.3
The pith
Stacking ensemble models trained on filtered NIR spectra quantify carbon and nitrogen in two soil types with RPD above 2.0.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Stacking ensembles that combine PLS, SVR, and Ridge regressions as base learners with linear regression as the meta-model, applied to Savitzky-Golay filtered NIR spectra after NIPALS-Huber outlier removal, achieve RPD greater than 2.0 and low overfitting for C and N prediction in Oxisols and Inceptisols under 10-fold, leave-one-out, and Kennard-Stone validation.
What carries the argument
Stacking ensemble with PLS, SVR, and Ridge base models plus linear regression meta-model, applied after Savitzky-Golay filtering and NIPALS-Huber outlier removal on portable MyNIR spectra.
Load-bearing premise
The NIR spectra contain enough information to predict C and N content without major unmodeled interference from moisture, particle size, or other soil factors.
What would settle it
A new collection of soil samples from the same types, measured on the same device and processed identically, that yields RPD below 1.5 on the trained stacking models would falsify the quantification claim.
Figures
read the original abstract
Near-Infrared (NIR) spectroscopy has emerged as a promising alternative to traditional soil analysis methods, offering advantages such as speed, low cost, and non-destructive testing. This work proposes a machine learning (ML) approach to calibrate predictive models for carbon (C) and nitrogen (N) content in Oxisols and Inceptisols, utilizing NIR spectral data acquired with a portable MyNIR device. Various preprocessing methods were evaluated, with the most effective being the Savitzky-Golay (SG) filter and a robust outlier removal method based on the Nonlinear Iterative Partial Least Squares (NIPALS) algorithm coupled with a Huber loss function. Multiple validation strategies were compared, including 10-fold cross-validation, leave-one-out, and holdout via the Kennard-Stone method, followed by standardization. Stacking ensemble learning models were employed, using Partial Least Squares (PLS), Support Vector Regression (SVR), and Ridge as base models, with linear regression as the meta-model. The models were evaluated using R2, Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Ratio of Performance Deviation (RPD) metrics. The performance gap between soil types suggests the influence of pedological characteristics. Furthermore, the models achieved an RPD > 2.0 with low overfitting, validating the potential of this approach for rapid C and N quantification. This study contributes to the optimization of sustainable agricultural practices, aligning with the demand for efficient and environmentally friendly analytical methods. The developed technique enables faster decision-making for producers and consultants based on organic matter content, fertility indicators, and nutrient availability.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that stacking ensembles (PLS, SVR, Ridge bases with linear regression meta-learner) trained on Savitzky-Golay filtered and NIPALS-Huber outlier-cleaned NIR spectra acquired with a portable MyNIR device achieve RPD > 2.0 for predicting carbon and nitrogen contents in Oxisol and Inceptisol soils. It compares multiple preprocessing pipelines and validation strategies (10-fold CV, LOOCV, Kennard-Stone holdout), reports performance gaps between soil types as evidence of pedological influence, and concludes that the approach enables rapid, low-cost quantification.
Significance. If the reported RPD values prove robust to moisture/particle-size confounding and to truly independent test sets, the work would support portable NIR + stacking ensembles as a practical alternative to laboratory wet chemistry for soil C/N, with direct relevance to precision agriculture and sustainable nutrient management. The systematic comparison of preprocessing and validation choices adds incremental practical guidance.
major comments (3)
- [§2.2] §2.2 (Preprocessing pipeline): No drying protocol, moisture measurement, or explicit correction (e.g., water-band subtraction or external parameter orthogonalization) is described despite moisture and particle-size effects being dominant, well-documented interferents in NIR soil spectra. The central claim that SG + NIPALS-Huber spectra encode C/N content therefore rests on an untested assumption that these covariates are uncorrelated with the target labels within the collected sample.
- [§2.3] §2.3 (Validation strategies): The Kennard-Stone holdout and all cross-validation folds are performed on the same spectral matrix after a single standardization step; no geographically or temporally independent test set is reported. This setup leaves open the possibility that reported RPD > 2.0 values reflect within-dataset correlations rather than generalizable predictive power.
- [Results] Results section (performance tables): Sample sizes (n per soil type), exact hyperparameter values for PLS (latent variables), SVR (C, ε, γ), and Ridge (α), and full residual distributions or bootstrap confidence intervals on RPD are not provided, preventing assessment of whether the claimed performance gap between Oxisols and Inceptisols is statistically reliable or merely descriptive.
minor comments (2)
- [Abstract] The abstract states that 'the performance gap between soil types suggests the influence of pedological characteristics' without a supporting statistical test; this interpretive sentence should be moved to the discussion or qualified.
- [§2.4] Notation for the stacking meta-learner and the exact definition of RPD (which RPD formula is used?) should be stated explicitly in the methods rather than assumed from prior literature.
Simulated Author's Rebuttal
We thank the referee for their thorough review and valuable suggestions. Below we provide point-by-point responses to the major comments. We have revised the manuscript to address the identified issues where feasible.
read point-by-point responses
-
Referee: [§2.2] §2.2 (Preprocessing pipeline): No drying protocol, moisture measurement, or explicit correction (e.g., water-band subtraction or external parameter orthogonalization) is described despite moisture and particle-size effects being dominant, well-documented interferents in NIR soil spectra. The central claim that SG + NIPALS-Huber spectra encode C/N content therefore rests on an untested assumption that these covariates are uncorrelated with the target labels within the collected sample.
Authors: We agree that moisture and particle-size effects are critical considerations in NIR soil analysis. The spectra in our study were acquired directly on field-collected samples using the portable MyNIR device without additional drying or correction steps, reflecting practical application conditions. We will revise section §2.2 to explicitly describe the sample collection and preparation procedures and add a discussion of potential confounding factors, including the assumption regarding uncorrelated covariates. This will also include recommendations for future work incorporating moisture correction techniques. revision: yes
-
Referee: [§2.3] §2.3 (Validation strategies): The Kennard-Stone holdout and all cross-validation folds are performed on the same spectral matrix after a single standardization step; no geographically or temporally independent test set is reported. This setup leaves open the possibility that reported RPD > 2.0 values reflect within-dataset correlations rather than generalizable predictive power.
Authors: The validation approaches used (10-fold CV, LOOCV, and Kennard-Stone holdout) follow standard practices for assessing model performance within the available dataset. We acknowledge that these do not constitute a fully independent external validation set from different geographic or temporal sources. We will revise the manuscript to explicitly note this limitation and emphasize that the reported performance applies to the studied soil types and conditions. Future extensions of this work will aim to include independent test sets. revision: yes
-
Referee: [Results] Results section (performance tables): Sample sizes (n per soil type), exact hyperparameter values for PLS (latent variables), SVR (C, ε, γ), and Ridge (α), and full residual distributions or bootstrap confidence intervals on RPD are not provided, preventing assessment of whether the claimed performance gap between Oxisols and Inceptisols is statistically reliable or merely descriptive.
Authors: We will update the Results section and associated tables to report the sample sizes for each soil type, the specific hyperparameter values selected for each base model, and include bootstrap-derived confidence intervals for the RPD metrics along with residual analysis to support the statistical reliability of the observed performance differences between soil types. revision: yes
Circularity Check
No circularity: standard empirical ML evaluation on held-out splits
full rationale
The paper describes collection of NIR spectra from soil samples, application of preprocessing (Savitzky-Golay, NIPALS-Huber), training of stacking ensembles (PLS, SVR, Ridge base learners with linear meta-learner), and reporting of R2/RMSE/MAE/RPD on 10-fold CV, LOOCV, and Kennard-Stone holdout splits. These steps constitute conventional supervised learning workflow; performance numbers are computed on samples excluded from each model's parameter estimation. No equations, uniqueness theorems, or self-citations are invoked that would make any reported quantity definitionally identical to the fitted inputs. The central claim (RPD > 2.0 validates the approach) rests on empirical results rather than reduction to prior definitions or self-referential fits.
Axiom & Free-Parameter Ledger
free parameters (2)
- Hyperparameters of base learners (PLS latent variables, SVR C/epsilon/gamma, Ridge alpha)
- Outlier threshold and NIPALS iteration settings
axioms (2)
- domain assumption NIR reflectance spectra contain sufficient quantitative information about soil C and N after standard preprocessing
- domain assumption Cross-validation and Kennard-Stone holdout provide unbiased estimates of generalization error for these models
Reference graph
Works this paper leans on
-
[1]
P. Nie, T. Dong, Y . He, F. Qu, Detection of soil nitrogen using near infrared sensors based on soil pretreatment and algorithms, Sensors 17 (2017) 1102
work page 2017
-
[2]
D. W. Pribyl, A critical review of the conventional soc to som conversion factor, Geoderma 156 (2010) 75–83
work page 2010
-
[3]
B. Van Raij, Fertilidade do solo e manejo de nutrientes; international plant nutrition institute (ipni): Piracicaba, s˜ao paulo, brazil, 2011, Google Scholar (2011) 420
work page 2011
-
[4]
Sparks, Environmental soil chemistry academic press, California
D. Sparks, Environmental soil chemistry academic press, California. USA (2003)
work page 2003
-
[5]
A. Segnini, L. M. d. Santos, W. T. L. d. Silva, L. Martin-Neto, C. E. Bo- rato, W. J. d. Melo, D. Bolonhezi, Estudo comparativo de m ´etodos para a determinac ¸˜ao da concentrac ¸˜ao de carbono em solos com altos teores de fe (latossolos), Qu´ımica Nova 31 (2008) 94–97. 61
work page 2008
-
[6]
H. G. d. Santos, P. K. T. Jacomine, L. H. C. d. Anjos, V . ´A. d. Oliveira, J. F. Lumbreras, M. R. Coelho, J. A. d. Almeida, J. C. d. Araujo Filho, H. N. Lima, F. A. Marques, J. B. d. Oliveira, T. J. F. Cunha, Sistema brasileiro de classificac ¸˜ao de solos, 6 ed., Embrapa, Bras ´ılia, DF, 2025. Dispon´ıvel em formato digital no portal da Embrapa
work page 2025
-
[7]
edu/agricultural-life-sciences/soil-orders/ inceptisols, 2026
University of Idaho, Inceptisols,https://www.uidaho. edu/agricultural-life-sciences/soil-orders/ inceptisols, 2026. Acessado em: 26 mai. 2026
work page 2026
-
[8]
R. V . Rossel, T. Behrens, E. Ben-Dor, D. Brown, J. A. M. Dematt ˆe, K. D. Shepherd, Z. Shi, B. Stenberg, A. Stevens, V . Adamchuk, et al., A global spectral library to characterize the world’s soil, Earth-Science Reviews 155 (2016) 198–230
work page 2016
-
[9]
R. B. A. Fernandes, I. A. d. Carvalho, E. S. Ribeiro, E. d. S. Mendonc ¸a, Comparison of different methods for the determination of total organic car- bon and humic substances in brazilian soils, Revista Ceres 62 (2015) 496– 501
work page 2015
-
[10]
E. Shamrikova, E. Vanchikova, B. Kondratenok, E. Lapteva, S. Kostrova, Problems and limitations of the dichromatometric method for measuring soil organic matter content: a review, Eurasian Soil Science 55 (2022) 861–867
work page 2022
-
[11]
D. W. Nelson, L. E. Sommers, Total carbon, organic carbon, and organic matter, in: D. L. Sparks, A. L. Page, P. A. Helmke, R. H. Loeppert, P. N. Soltanpour, M. A. Tabatabai, C. T. Johnston, M. E. Sumner (Eds.), Methods of Soil Analysis: Part 3 Chemical Methods, number 5 in SSSA Book Se- 62 ries, Soil Science Society of America and American Society of Agro...
work page 1996
-
[12]
Jones Jr, Kjeldahl method for nitrogen determination, Micro-Macro Pub- lishing, Inc., 1991
J. Jones Jr, Kjeldahl method for nitrogen determination, Micro-Macro Pub- lishing, Inc., 1991
work page 1991
-
[13]
J. M. Bremner, Determination of nitrogen in soil by the Kjeldahl method, The Journal of Agricultural Science 55 (1960) 11–33
work page 1960
-
[14]
P. S ´aez-Plaza, T. Michałowski, M. J. Navas, A. G. Asuero, S. Wybraniec, An overview of the Kjeldahl method of nitrogen determination. Part I. Early history, chemistry of the procedure, and titrimetric finish, Critical Reviews in Analytical Chemistry 43 (2013) 178–223
work page 2013
- [15]
- [16]
-
[17]
M. Zareef, M. Arslan, W. Ahmad, M. M. Hassan, M. Shoaib, Q. Ouyang, H. Li, M. M. Hashim, S. Javaria, Q. Chen, Applications of benchtop and portable spectroscopy techniques for food quality monitoring, Spectrochim- ica Acta Part A: Molecular and Biomolecular Spectroscopy 344 (2026) 126713
work page 2026
-
[18]
J. T. Oliva, V . Kieling, F. Rossi, E. O. Rodrigues, G. Guarneri, L. dos San- tos Tonial, Comparison between portable and bench-top near-infrared spec- 63 troscopy for corn silage characterization using partial least square and sup- port vector regression methods, Journal of Chemometrics 39 (2025) e70073. E70073 CEM-25-0139
work page 2025
-
[19]
R. A. Viscarra Rossel, R. N. McGlynn, A. B. McBratney, Determination of reference values for nir spectra of soils, Geoderma 137 (2006) 70–82
work page 2006
-
[20]
J. M. Soriano-Disla, L. J. Janik, D. J. Allen, M. J. McLaughlin, Evaluation of the performance of portable visible-infrared instruments for the prediction of soil properties, Biosystems Engineering 161 (2017) 24–36
work page 2017
- [21]
-
[22]
B. Stenberg, R. A. Viscarra Rossel, A. M. Mouazen, J. Wetterlind, Visible and near infrared spectroscopy in soil science, Advances in Agronomy 107 (2010) 163–215
work page 2010
- [23]
-
[24]
J. Workman Jr, L. Weyer, Practical guide to interpretive near-infrared spec- troscopy, CRC press, 2007
work page 2007
-
[25]
Z. Wang, W. Huang, J. Li, S. Liu, S. Fan, Assessment of protein content and insect infestation of maize seeds based on on-line near-infrared spectroscopy 64 and machine learning, Computers and Electronics in Agriculture 211 (2023) 107969
work page 2023
-
[26]
J. Liu, X. Luo, D. Zhang, C. Wang, Z. Chen, X. Zhao, Rapid determination of rice protein content using near-infrared spectroscopy coupled with feature wavelength selection, Infrared Physics & Technology 135 (2023) 104969
work page 2023
-
[27]
M. Sisouane, M. Cascant, S. Tahiri, S. Garrigues, M. El Krati, G. E. K. Boutchich, M. Cervera, M. de La Guardia, Prediction of organic carbon and total nitrogen contents in organic wastes and their composts by infrared spectroscopy and partial least square regression, Talanta 167 (2017) 352– 358
work page 2017
-
[28]
F. Bedin, M. Faust, G. Guarneri, T. S. Assmann, C. Lafay, L. Soares, P. de Oliveira, L. dos Santos Tonial, NIR associated to PLS and SVM for fast and non-destructive determination of C, N, P, and K contents in poultry litter, Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy 245 (2021) 118834
work page 2021
-
[29]
A. Sharififar, K. Singh, E. Jones, F. I. Ginting, B. Minasny, Evaluating a low-cost portable nir spectrometer for the prediction of soil organic and total carbon using different calibration models, Soil Use and Management 35 (2019) 607–616
work page 2019
-
[30]
L. Benedet, W. M. Faria, S. H. G. Silva, M. Mancini, J. A. M. Dematt ˆe, L. R. G. Guilherme, N. Curi, Soil texture prediction using portable x-ray fluorescence spectrometry and visible near-infrared diffuse reflectance spec- troscopy, Geoderma 376 (2020) 114553. 65
work page 2020
-
[31]
Y . Tang, E. Jones, B. Minasny, Evaluating low-cost portable near infrared sensors for rapid analysis of soils from south eastern australia, Geoderma Regional 20 (2020) e00240
work page 2020
- [32]
-
[33]
N. M. Dhawale, V . Adamchuk, S. Prasher, R. V . Rossel, A. Ismail, Evalu- ation of two portable hyperspectral-sensor-based instruments to predict key soil properties in canadian soils, Sensors (Basel, Switzerland) 22 (2022)
work page 2022
-
[34]
J. K. Carvalho, J. M. Moura-Bueno, R. Ramon, T. F. Almeida, G. Naibo, A. P. Martins, L. S. Santos, C. Gianello, T. Tiecher, Combining different pre- processing and multivariate methods for prediction of soil organic matter by near infrared spectroscopy (nirs) in southern brazil, Geoderma Regional 29 (2022) e00530
work page 2022
- [35]
-
[36]
M. Saberioon, A. Gholizadeh, A. Ghaznavi, S. Chabrillat, V . Khosravi, En- hancing soil organic carbon prediction of lucas soil database using deep learning and deep feature selection, Computers and Electronics in Agri- culture 227 (2024) 109494
work page 2024
-
[37]
Z. Xu, X. Zhao, X. Guo, J. Guo, Deep learning application for predicting soil 66 organic matter content by vis-nir spectroscopy, Computational Intelligence and Neuroscience 2019 (2019) 3563761
work page 2019
-
[38]
G. Gozukara, A. E. Hartemink, J. Huang, J. A. M. Dematt ˆe, Prediction accuracy of pxrf, mir, and vis-nir spectra for soil properties—a review, Soil Science Society of America Journal 89 (2025) e70028
work page 2025
-
[39]
A. Walkley, I. A. Black, An examination of the degtjareff method for deter- mining soil organic matter, and a proposed modification of the chromic acid titration method, Soil science 37 (1934) 29–38
work page 1934
-
[40]
F. C. da Silva, Manual de an ´alises qu´ımicas de solos, plantas e fertilizantes. (2009)
work page 2009
-
[41]
M. Tedesco, C. Gianello, C. Bissani, H. Bohnen, S. V olkweiss, An ´alise de solo, plantas e outros materiais (in Portuguese), Boletim t ´ecnico 5 (1995)
work page 1995
- [42]
-
[43]
V . Sharma, A study on data scaling methods for machine learning, Interna- tional Journal for Global Academic & Scientific Research 1 (2022) 31–42
work page 2022
-
[44]
T. Isaksson, T. Næs, The effect of multiplicative scatter correction (msc) and linearity improvement in nir spectroscopy, Applied Spectroscopy 42 (1988) 1273–1284. 67
work page 1988
- [45]
-
[46]
A. F. Goetz, G. Vane, J. E. Solomon, B. N. Rock, Imaging spectrometry for earth remote sensing, science 228 (1985) 1147–1153
work page 1985
-
[47]
D. A. Skoog, F. J. Holler, S. R. Crouch, Principles of Instrumental Analysis, 7th ed., Cengage Learning, Boston, MA, 2017
work page 2017
- [48]
- [49]
-
[50]
S. Wold, K. Esbensen, P. Geladi, Principal component analysis, Chemomet- rics and intelligent laboratory systems 2 (1987) 37–52
work page 1987
-
[51]
A. Savitzky, M. Golay, Smoothing and differentiation of data by simplified least squares procedures, Analytical chemistry 36 (1964) 1627–1639
work page 1964
-
[52]
E. J. Bjerrum, M. Glahder, T. Skov, Data augmentation of spectral data for convolutional neural network (cnn) based deep chemometrics, arXiv preprint arXiv:1710.01927 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
- [53]
-
[54]
S. Raschka, V . Mirjalili, Python machine learning: Machine learning and deep learning with Python, scikit-learn, and TensorFlow 2, Packt publishing ltd, 2019
work page 2019
-
[55]
R. d. A. Ferreira, G. Teixeira, L. A. Peternelli, Kennard-stone method out- performs the random sampling in the selection of calibration samples in snps and nir data, Ci ˆencia Rural 52 (2021) e20201072
work page 2021
-
[56]
R. W. Kennard, L. A. Stone, Computer aided design of experiments, Tech- nometrics 11 (1969) 137–148
work page 1969
-
[57]
T. F. Boucher, M. V . Ozanne, M. L. Carmosino, M. D. Dyar, S. Mahadevan, E. A. Breves, K. H. Lepore, S. M. Clegg, A study of machine learning regression methods for major elemental analysis of rocks using laser-induced breakdown spectroscopy, Spectrochimica Acta Part B: Atomic Spectroscopy 107 (2015) 1–10
work page 2015
-
[58]
M. Stone, Cross-validatory choice and assessment of statistical predictions, Journal of the royal statistical society: Series B (Methodological) 36 (1974) 111–133
work page 1974
-
[59]
D. C. Montgomery, G. C. Runger, Applied statistics and probability for en- gineers, John wiley & sons, 2010
work page 2010
-
[60]
M. Khan, S. Noor, Performance analysis of regression-machine learning algorithms for predication of runoff time, Agrotechnology 8 (2019) 1–12
work page 2019
-
[61]
E. D. Louw, K. I. Theron, Robust prediction models for quality parameters in japanese plums (prunus salicina l.) using nir spectroscopy, Postharvest Biology and Technology 58 (2010) 176–184. 69
work page 2010
-
[62]
C. J. Willmott, K. Matsuura, Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model perfor- mance, volume 30, Inter-Research, 2005
work page 2005
-
[63]
T. Chai, R. R. Draxler, Root mean square error (rmse) or mean absolute error (mae)?, Geoscientific Model Development 7 (2014) 1247–1250
work page 2014
- [64]
-
[65]
K. H. Esbensen, P. Geladi, A. Larsen, The rpd myth. . . , NIR news 25 (2014) 24–28. 70
work page 2014
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.