6KSFx Synth Dataset

Joshua Reiss; Nelly Garcia

REVIEW 5 major objections 4 minor 24 references

6KSFx Synth Dataset

T0 review · 5 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper presents a public dataset of 6,000 synthetic audio samples divided evenly across 30 sound-effect categories, with the synthesis methods behind each category documented.

desk verdict A genuinely useful dataset idea undermined by avoidable internal contradictions and label errors that make the release impossible to trust as described. read the letter →

arxiv 2501.17198 v1 pith:UXTBVKUM submitted 2025-01-27 cs.SD eess.ASphysics.data-an

Nelly Garcia , Joshua Reiss This is my paper · ORCID

classification cs.SDeess.ASphysics.data-an

keywords proceduralaudiosyntheticsoundeffectsdigitalFoleysynthesissound-effectsdatasetclassificationbenchmark6KSFx

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that a large, balanced, and publicly available synthetic sound-effects corpus is now a practical resource: 6,000 computer-generated samples, five seconds each, spread across 30 everyday sound categories such as applause, rain, footsteps, and rocket. Each category contains 200 synthetic samples and is paired with a recorded version under an even/odd label scheme, and the paper documents which synthesis methods were used for each category. The purpose is to remove the biggest barrier to procedural audio research, the absence of a labeled public dataset with known synthesis provenance. If the dataset is as described, researchers and sound designers gain a common benchmark for comparing synthetic effects with recordings and for training and testing audio classifiers. The synthetic samples were processed with reverb and equalization so that comparisons with real recordings are meant to be fair.

What carries the argument

The carrying object is the pairing scheme with its label arithmetic and the category-to-method metadata table. Real and synthetic renditions of each sound category receive consecutive labels, even for the recording and odd for the synthesis, so any two adjacent labels are directly comparable. The paper then assigns each odd label one to three methods from its eight-method taxonomy, giving users a route from a sound file back to the code-level recipe that produced it. The preprocessing chain, procedural generation followed by reverb and equalization, is the mechanism intended to put synthetic and recorded samples on equal footing for listening tests or machine-learning evaluation.

What would settle it

Download the public release, count the files in each of the 30 categories, and compare every filename's numeric label with the parity claimed in Table 1; a category with anything other than 200 files, or a file whose label breaks the even/odd pairing, would show that the dataset description is not accurate.

Watch

Extended reading notes

Core claim

On the paper's own terms, the contribution is the dataset itself rather than a new synthesis technique. The authors generated synthetic audio algorithmically, treated it with reverb and equalization, and organized it into 30 categories of 200 samples at a fixed format of 5 seconds, 44.1 kHz, mono, and 16 bits. A companion table maps each category to one to three synthesis methods chosen from additive, subtractive, granular, physical modeling, physically informed, modal, signal modeling, and frequency modeling. The resulting release is meant to make procedural audio measurable: the same category always appears as a real and a synthetic pair, so quality, realism, and method effectiveness can be compared without confounding on content.

Load-bearing premise

The load-bearing premise is that the files in the public download match the paper's descriptions: 200 samples per category, the stated format, the even/odd real-synthetic labeling, and the synthesis methods listed in Table 2.

Editorial extensions

If this is right

A sound-effects classifier can be trained and tested on 30 balanced classes, with the even/odd pairs separating real from synthetic audio within each class.
Synthesis-method comparisons can be made within a category, because the metadata explains which technique generated each label.
The fixed 5-second mono format at 44.1 kHz lets researchers use the corpus as a benchmark without additional normalization.
Procedural audio researchers can cite one public release instead of relying on private or defunct sample libraries.
Sound designers can use the documented recipes to generate new variations of a desired effect rather than recording or licensing it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same even/odd pairing could support a synthetic-audio detection task, in which a model learns to tell generated effects from recorded ones under matched conditions; the paper does not itself train such a detector.
Editorial inference: one could extend the release by asking listeners to rate realism per category and then correlate those ratings with the tabulated synthesis methods, which would turn the dataset into a perceptual benchmark rather than just a collection.
Editorial inference: because the method labels were assigned by reading code, a natural check is to see whether the documented recipes are acoustically distinguishable, for instance by unsupervised clustering of the samples; the paper reports no such analysis.
Editorial inference: the same dataset structure could be scaled to finer-grained subcategories, such as different surfaces or intensities, to support harder evaluation tasks than 30-way classification.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper has a real deliverable in mind—a public, labeled synthetic sound-effects dataset—but the manuscript as written makes it impossible to trust that deliverable. The internal contradictions and labeling errors are concrete and need to be fixed before anyone uses this as a resource.

What's genuinely new: 6,000 synthetic samples released on Zenodo, with per-category synthesis-method labels, fills a documented gap—earlier synthetic collections (Moffat and Reiss) were not publicly accessible, and mainstream datasets are real recordings. That's a useful artifact for procedural audio evaluation and for machine learning work on sound effects.

What's done well: the pairing of real and synthetic samples across 30 categories is a sensible design for comparative evaluation; the synthesis-method taxonomy is grounded in prior work and applied to each category; and the authors are upfront that classifications are subjective.

Now the soft spots, and they're not minor. The text contradicts itself in three places. The Introduction promises a public dataset with both synthetic and pre-recorded samples; Section 3 says real samples are not released; the Conclusion claims the dataset 'integrates' both. That's a load-bearing confusion about what the artifact actually is. Section 3 also says '30 samples per category' and then says each category accounts for 200 samples—a direct numeric conflict. Table 1, the only map between filenames and categories, skips label 36 (Whoosh is 37/38 instead of 36/37) and assigns label 42 twice. Table 2 uses different category names ('Debris Glass' vs 'Glass Debris', 'Boat Engine' vs 'Boat', 'Fire' vs 'Fire Embers'). No manifest, checksum, or per-category count is supplied to bridge the paper and the Zenodo files.

These aren't cosmetic issues. The paper's stated purpose is to enable evaluation frameworks that depend on correct labels. If the released files follow Table 1, the annotations for Whoosh and the two footstep categories are wrong; if the files are correct, then Table 1 is wrong and the paper can't be used as a map. Either way, the resource isn't usable as described.

The core claim—that a dataset exists—is plausible and probably true, but the paper does not give us a way to verify it. That said, the flaws are fixable. The authors need to reconcile the text, fix the label tables, and add a manifest with counts and either checksums or a verification script.

Who this is for: researchers working on procedural audio, sound effect classification/generation, and audio evaluation benchmarks. They'd get value once the dataset is properly documented.

My recommendation: send it to peer review, but the reviewers should insist on a revision that clears up these contradictions and provides the verification artifacts. A dataset paper's worth is in the artifact; right now the paper undersells it with avoidable errors.

Referee Report

5 major / 4 minor

Summary. The paper introduces '6KSFx Synth Dataset', a claimed public dataset of 6,000 synthetic audio samples generated with procedural audio techniques, intended to support research on sound-synthesis evaluation and classification. The manuscript describes 30 sound categories, each represented by real and synthetic samples, with a synthesis-method taxonomy provided in Table 2. The dataset is said to be released on Zenodo, with preprocessing code on GitHub. The central claim is that a downloadable, labeled corpus of 6,000 procedural audio samples exists and is publicly available.

Significance. If the dataset is released as described and the labels are correct, this would be a useful community resource: procedural audio research has few public synthetic datasets, and a corpus with per-category synthesis metadata could support reproducible evaluation, machine-learning benchmarks, and comparisons of synthesis methods. The paper also provides explicit Zenodo and GitHub links and a descriptive taxonomy of synthesis methods, which is helpful for orienting users. However, the contribution's value is entirely dependent on the integrity and usability of the released artifact: the paper contains no derivations to verify, and the supporting claims about the dataset's organization contain internal contradictions that must be resolved before the resource can be used reliably.

major comments (5)

[Section 3, Table 1] Table 1 has demonstrable label-numbering errors that break the mapping from file names to categories: the Whoosh category is assigned labels 37 and 38, but the preceding Jet row ends at 35, so label 36 is missing and the even/odd real-synthetic pairing is shifted. More seriously, Concrete Footsteps is labeled 41/42 and Wood Footsteps is labeled 42/43, so label 42 is duplicated and the label-uniqueness assumption underlying the filename scheme ('Name of the sound category' - 'Sample number' - 'Label') fails. The authors must correct the table or, if the released files use different labels, explicitly state that Table 1 is not the key to the release and provide the correct mapping.
[Section 3] The sample counts are contradictory. The text first states that the dataset was constructed using 12,000 five-second samples evenly distributed across 30 categories, then says the public release is 6,000 synthetic samples 'divided into 30 samples per category', and then states that 'each category accounts for 1.8% of the dataset (200 samples per category)'. If there are 200 samples per category, 30 categories give 6,000 samples, so the total cannot also be 12,000 with the same per-category balance unless the real subset is also 200 per category. The claim of balanced distribution cannot be interpreted as written; please state the exact number of files per category and per subset.
[Section 1 bullet list; Section 3; Section 4] The paper contradicts itself on what is publicly available. The Introduction bullet says the dataset provides 'a public dataset containing both synthetic and pre-recorded samples', while Section 3 says the real samples are not publicly available due to copyright restrictions and only links to providers are given. The Conclusion then says the dataset 'integrat[es] both real and synthetic sounds'. These statements describe incompatible release scopes; the authors should state unambiguously whether the released Zenodo artifact contains only synthetic samples, and whether the real samples are available at all.
[Tables 1 and 2] The category names in Table 2 do not match those in Table 1: 'Debris Glass' vs 'Glass Debris', 'Boat Engine' vs 'Boat', 'Fire' vs 'Fire Embers', and 'Bounce Rubber' vs 'Bounce (Rubber)'. Because the synthesis-method metadata in Table 2 is meant to be joined to the released files via the Table 1 labels, this naming mismatch makes the method annotations ambiguous. The authors should align all category names across Table 1, Table 2, Figure 1, and the actual file names on Zenodo.
[Section 3] No manifest, file count, checksum, or programmatic verification is provided to confirm that the Zenodo artifact matches the tables. Given that Table 1 already contains label errors and the count text is inconsistent, the absence of a verifiable manifest is a load-bearing omission: a reader cannot tell whether the released files follow the erroneous Table 1 or a corrected version. Please include a manifest or a verification script that checks file counts, labels, durations, and sampling rates against the corrected metadata.

minor comments (4)

[Tables 1 and 2] The word 'Footsteps' appears as 'F ootsteps' in both tables; please fix the spacing.
[Abstract and Section 1] 'pre recorded' should be 'pre-recorded' throughout, and 'syntheis' in Reference [20] should be 'synthesis'.
[Section 3, Figure 1] The text says 'the distribution shown in 1' but should say 'shown in Figure 1'.
[References] Reference [23] cites a paper on microwave resonator filter synthesis, which appears unrelated to sound synthesis; please replace it with a relevant physical-modeling or resonator synthesis reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper presents an external dataset artifact and descriptive metadata, with no derived claim that reduces to its own inputs.

full rationale

The paper's central claim is the release of 6,000 synthetic audio samples on Zenodo, an externally downloadable artifact, together with category labels and synthesis-method annotations. There is no fitted parameter, no predictive equation, and no derivation whose conclusion is assumed in its premises. The synthesis taxonomy is attributed to prior work including a co-author ([13], Menexopoulos, Pestana, and Reiss), but the paper explicitly states that the method classifications come from code inspection ('we analyzed the underlying code for each sound effect to identify the specific synthesis methods used') and even cautions that 'classifications can be subjective'; the citation therefore supplies a descriptive naming framework, not the content of the dataset. The internal inconsistencies noted in the paper, such as the missing label 36 and duplicate label 42 in Table 1, the contradictory per-category counts in Section 3, and the split between the Introduction's promise of both synthetic and pre-recorded samples and Section 3's statement that real samples are not publicly available, are data-quality and reproducibility concerns, not instances of circular reasoning. Because the paper's headline claim is a public release rather than a numerically derived result, and because no load-bearing step reduces by construction to the paper's own definitions or citations, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim is an artifact release, so the load-bearing assumptions are about the generation engine's correctness, the taxonomy's application, and the fidelity of the uploaded files. There are no fitted free parameters or new postulated entities.

assumptions (3)

domain assumption The Nemisindo procedural audio engine produces samples that correctly match the 30 named sound categories.
The dataset's category labels depend on the engine's output matching the naming in Table 1; no verification is provided in the paper.
domain assumption The synthesis-method taxonomy from Menexopoulos et al. 2023 is correctly applied to each sound model.
Table 2's classifications are based on this framework, and incorrect assignment would mislead users of the metadata.
ad hoc to paper The files uploaded to Zenodo follow the naming scheme and label numbers given in Table 1.
The paper provides no listing or checksum of the released files; the referenced labels exhibit errors in the table itself.

how reviews work

0 comments

Cite this review

Pith. "Pith review of 6KSFx Synth Dataset." pith.science (2026). https://pith.science/paper/UXTBVKUM

@misc{pith2026250117198,
  author       = {Pith},
  title        = {Pith review of: 6KSFx Synth Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UXTBVKUM}},
  note         = {Machine review of arXiv:2501.17198}
}

read the original abstract

Procedural audio, often referred to as "digital Foley", generates sound from scratch using computational processes. It represents an innovative approach to sound-effects creation. However, the development and adoption of procedural audio has been constrained by a lack of publicly available datasets and models, which hinders evaluation and optimization. To address this important gap, this paper presents a dataset of 6000 synthetic audio samples specifically designed to advance research and development in sound synthesis within 30 sound categories. By offering a description of the diverse synthesis methods used in each sound category and supporting the creation of robust evaluation frameworks, this dataset not only highlights the potential of procedural audio, but also provides a resource for researchers, audio developers, and sound designers. This contribution can accelerate the progress of procedural audio, opening up new possibilities in digital sound design.

Figures

Figures reproduced from arXiv: 2501.17198 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 24 canonical work pages

[1]

Specsingan: Sound effect variation synthesis using single-image gans

Adrián Barahona-Ríos and Tom Collins. Specsingan: Sound effect variation synthesis using single-image gans. The Computing Research Repository, 2021

work page 2021
[2]

Procedural audio in computer games using motion controllers: An evaluation on the effect and perception

Niels Böttcher, Héctor Martínez, and Stefania Serafin. Procedural audio in computer games using motion controllers: An evaluation on the effect and perception. International Journal of Computer Games Technology, 2013:1–16, 01 2013

work page 2013
[3]

Vggsound: A large-scale audio-visual dataset

Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. Vggsound: A large-scale audio-visual dataset. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 721–725, 2020

work page 2020
[4]

A proposal for foley sound synthesis challenge, 07 2022

Keunwoo Choi, Sangshin Oh, Minsung Kang, and Brian McFee. A proposal for foley sound synthesis challenge, 07 2022

work page 2022
[5]

Clotho: an audio captioning dataset

Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. Clotho: an audio captioning dataset. pages 736–740, 05 2020

work page 2020
[6]

Designing sound

Andy Farnell. Designing sound. Mit Press, 08 2010

work page 2010
[7]

Vampnet: Music generation via masked acoustic token modeling, 2023

Hugo Flores Garcia, Prem Seetharaman, Rithesh Kumar, and Bryan Pardo. Vampnet: Music generation via masked acoustic token modeling, 2023

work page 2023
[8]

Sketch2sound: Control- lable audio generation via time-varying signals and sonic imitations, 2024

Hugo Flores García, Oriol Nieto, Justin Salamon, Bryan Pardo, and Prem Seetharaman. Sketch2sound: Control- lable audio generation via time-varying signals and sonic imitations, 2024

work page 2024

Show all 24 references

[9]

Audio set: An ontology and human-labeled dataset for audio events

Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASS...

2017
[10]

From sound modeling to analysis-synthesis of sounds

Richard Kronland-Martinet, Philippe Guillemain, and Sølvi Ystad. From sound modeling to analysis-synthesis of sounds. Universitat Pompeu Fabra, 2001

2001
[11]

Higher-order frequency modulation synthesis, 05 2023

Victor Lazzarini and Joseph Timoney. Higher-order frequency modulation synthesis, 05 2023

2023
[12]

Sketchsynth: Cross-modal control of sound synthesis

Sebastian Löbbers, Louise Thorpe, and György Fazekas. Sketchsynth: Cross-modal control of sound synthesis. In EvoMUSART@EvoStar, 2023

2023
[13]

The state of the art in procedural audio

Dimitris Menexopoulos, Pedro Pestana, and Joshua Reiss. The state of the art in procedural audio. Journal of the Audio Engineering Society, 71:825–847, 12 2023

2023
[14]

Objective evaluations of synthesised environmental sounds

David Moffat and Joshua Reiss. Objective evaluations of synthesised environmental sounds. 09 2018

2018
[15]

Perceptual evaluation of synthesized sound effects

David Moffat and Joshua Reiss. Perceptual evaluation of synthesized sound effects. ACM Transactions on Applied Perception, 15:1–19, 04 2018

2018
[16]

Sound Effect Synthesis, pages 274–299

David Moffat, Rod Selfridge, and Joshua Reiss. Sound Effect Synthesis, pages 274–299. 06 2019

2019
[17]

The computer music tutorial

Curtis Roads. The computer music tutorial. 1996. 6 6KSFx Synth Dataset

1996
[18]

A dataset and taxonomy for urban sound research

Justin Salamon, Christopher Jacoby, and Juan Bello. A dataset and taxonomy for urban sound research. 11 2014

2014
[19]

Antialiasing oscillators in subtractive synthesis

Vesa Välimäki and Antti Huovilainen. Antialiasing oscillators in subtractive synthesis. Signal Processing Magazine, IEEE, 24:116 – 125, 04 2007

2007
[20]

On the limits of real-time physical modelling syntheis with a modular environment

Craig Webb and Stefan Bilbao. On the limits of real-time physical modelling syntheis with a modular environment. 11 2015

2015
[21]

Procedural Audio in Video Games , pages 1483–1487

Matthew Yee-King and Igor Dall’Avanzi. Procedural Audio in Video Games , pages 1483–1487. Springer International Publishing, 2024

2024
[22]

Visual to sound: Generating natural sound for videos in the wild

Zhou Yipin, Wang Zhaowen, Fang Chen, Bui Trung, and Berg Tamara L. Visual to sound: Generating natural sound for videos in the wild. The Computing Research Repository, 2017

2017
[23]

General synthesis method for dispersively coupled resonator filters with cascaded topologies

Yan Zhang, Fabien Seyfert, Smain Amari, Martine Olivi, and Ke-Li Wu. General synthesis method for dispersively coupled resonator filters with cascaded topologies. IEEE Transactions on Microwave Theory and Techniques, 12 2020

2020
[24]

DAFX: Digital Audio Effects

Udo Zolzer. DAFX: Digital Audio Effects. Wiley Publishing, 2nd edition, 2011. 7

2011

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Specsingan: Sound effect variation synthesis using single-image gans

Adrián Barahona-Ríos and Tom Collins. Specsingan: Sound effect variation synthesis using single-image gans. The Computing Research Repository, 2021

work page 2021

[2] [2]

Procedural audio in computer games using motion controllers: An evaluation on the effect and perception

Niels Böttcher, Héctor Martínez, and Stefania Serafin. Procedural audio in computer games using motion controllers: An evaluation on the effect and perception. International Journal of Computer Games Technology, 2013:1–16, 01 2013

work page 2013

[3] [3]

Vggsound: A large-scale audio-visual dataset

Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. Vggsound: A large-scale audio-visual dataset. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 721–725, 2020

work page 2020

[4] [4]

A proposal for foley sound synthesis challenge, 07 2022

Keunwoo Choi, Sangshin Oh, Minsung Kang, and Brian McFee. A proposal for foley sound synthesis challenge, 07 2022

work page 2022

[5] [5]

Clotho: an audio captioning dataset

Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. Clotho: an audio captioning dataset. pages 736–740, 05 2020

work page 2020

[6] [6]

Designing sound

Andy Farnell. Designing sound. Mit Press, 08 2010

work page 2010

[7] [7]

Vampnet: Music generation via masked acoustic token modeling, 2023

Hugo Flores Garcia, Prem Seetharaman, Rithesh Kumar, and Bryan Pardo. Vampnet: Music generation via masked acoustic token modeling, 2023

work page 2023

[8] [8]

Sketch2sound: Control- lable audio generation via time-varying signals and sonic imitations, 2024

Hugo Flores García, Oriol Nieto, Justin Salamon, Bryan Pardo, and Prem Seetharaman. Sketch2sound: Control- lable audio generation via time-varying signals and sonic imitations, 2024

work page 2024