pith. machine review for the scientific record. sign in

arxiv: 2604.21682 · v1 · submitted 2026-04-23 · 📡 eess.AS

Recognition: unknown

PHOTON: Non-Invasive Optical Tracking of Key-Lever Motion in Historical Keyboard Instruments

John Ashley Burgoyne, Noah Jaffe

Authors on Pith no claims yet

Pith reviewed 2026-05-08 13:11 UTC · model grok-4.3

classification 📡 eess.AS
keywords optical sensingkey-lever motionhistorical keyboard instrumentsnon-invasive trackingperformance captureMIDI outputharpsichordclavichord
0
0 comments X

The pith

Reflective optical sensors track the full vertical motion of keys in historical keyboard instruments without mechanical contact or alteration.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces PHOTON as a sensing approach that places optical detectors under the keys of older instruments to record how far and how fast each lever actually travels during play. This records the combined effects of the player's touch and the instrument's own changing mechanical resistance, yielding continuous data on position, timing, and touch details. The design fits the irregular shapes and tight spaces found in harpsichords, clavichords, and early pianos, works on instruments with varying numbers of keys and multiple manuals, and outputs MIDI signals in real time. The full hardware and software are released openly so others can build or adapt it. A reader would care because the method lets researchers examine real expressive playing and instrument responses on preserved historical mechanisms instead of relying on modern substitutes or indirect estimates.

Core claim

PHOTON tracks the vertical displacement of the key lever itself, capturing motion shaped by both performer input and the instrument's mechanically imposed, time-varying load. Reflective optical sensors mounted beneath the distal end of each lever provide continuous displacement, timing, and articulation data without interfering with the action. The modular, low-profile architecture accommodates the diverse geometries, limited clearances, and non-standard layouts of harpsichords, clavichords, and early fortepianos while enabling high-resolution, low-latency sensing across multiple manuals and variable key counts, with real-time MIDI output and open-source release for replication.

What carries the argument

Reflective optical sensors placed beneath the distal end of each key lever, which measure vertical displacement to produce continuous profiles of motion under combined player and instrument loads.

If this is right

  • Real-time MIDI output becomes available directly from unaltered historical instruments.
  • Empirical studies of expressive gesture and human-instrument interaction can draw on authentic mechanical responses.
  • Instrument-specific MIDI corpora can be constructed from real historical mechanisms rather than modern approximations.
  • The open-source hardware and software lower the barrier for replication and further development by other researchers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The motion data could improve digital models that simulate the dynamic response of historical actions inside virtual instruments.
  • Comparative studies of performance practice across eras could use consistent measurements taken from preserved instruments.
  • Combining the optical tracks with other non-contact sensors might allow reconstruction of the full internal action chain in one session.
  • The system could support training tools that give players immediate visual or auditory feedback on timing and touch when learning historical techniques.

Load-bearing premise

Reflective optical sensors mounted under the keys can deliver high-resolution, low-latency displacement data across varied instrument sizes and layouts without mechanical interference or calibration changes that affect playability.

What would settle it

Mounting and operating the sensors on an actual historical instrument and observing either altered key travel distance, increased resistance, delayed response, or data that fails to resolve fine timing and articulation differences during normal playing.

Figures

Figures reproduced from arXiv: 2604.21682 by John Ashley Burgoyne, Noah Jaffe.

Figure 1
Figure 1. Figure 1: Measurement of key-lever motion under dynamic sounding load. view at source ↗
Figure 2
Figure 2. Figure 2: Harpsichord manual with PHOTON module. invasive arrangement that is more readily extended to double￾manual instruments; it also avoids the need for optical baffles between adjacent sensors and is compatible with substantially tighter installation spaces. More closely related to the present work, Schmidt et al. place sensors beneath the distal ends of the keys to capture key position [15]. PHOTON adopts the… view at source ↗
Figure 3
Figure 3. Figure 3: Representative PHOTON motion traces showing view at source ↗
Figure 4
Figure 4. Figure 4: Sensing mode of the VCNT2025X01 sensor. double-manual configuration an additional, higher feature ap￾pears near 7.0 mm, corresponding to the second plucking event produced by the staggered, coupled actions. The displacement axis is obtained by mapping per-sensor ADC readings between two mechanically measured reference positions, with rest defined as 0 mm displacement and full key travel measured as 9 mm; i… view at source ↗
Figure 5
Figure 5. Figure 5: Schematic excerpt from the sensor array, showing view at source ↗
Figure 6
Figure 6. Figure 6: Printed circuit board assembly (PCBA) layout ex view at source ↗
Figure 7
Figure 7. Figure 7: PHOTON main controller board featuring the view at source ↗
read the original abstract

This paper introduces PHOTON (PHysical Optical Tracking of Notes), a non-invasive optical sensing system for measuring key-lever motion in historical keyboard instruments. PHOTON tracks the vertical displacement of the key lever itself, capturing motion shaped by both performer input and the instrument's mechanically imposed, time-varying load. Reflective optical sensors mounted beneath the distal end of each lever provide continuous displacement, timing, and articulation data without interfering with the action. Unlike existing optical systems designed for modern pianos, PHOTON accommodates the diverse geometries, limited clearances, and non-standard layouts of harpsichords, clavichords, and early fortepianos. Its modular, low-profile architecture enables high-resolution, low-latency sensing across multiple manuals and variable key counts. Beyond performance capture, PHOTON provides real-time MIDI output and supports empirical study of expressive gesture, human-instrument interaction, and the construction of instrument-specific MIDI corpora using real historical mechanisms. The complete system is released as open-source hardware and software, from schematics and PCB layouts developed in KiCad to firmware written in CircuitPython, lowering the barrier to adoption, replication, and extension.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The manuscript introduces PHOTON, a non-invasive optical sensing system for tracking vertical key-lever displacement in historical keyboard instruments (harpsichords, clavichords, early fortepianos). Reflective optical sensors mounted beneath the distal end of each lever capture continuous displacement, timing, and articulation data shaped by performer input and the instrument's time-varying mechanical load. The modular, low-profile design accommodates diverse geometries, limited clearances, non-standard layouts, and multiple manuals without mechanical interference, provides real-time MIDI output, and supports studies of expressive gesture and human-instrument interaction. The full hardware (KiCad schematics and PCB layouts) and software (CircuitPython firmware) stack is released as open source.

Significance. If the performance claims are substantiated, PHOTON would fill a clear gap by enabling non-invasive, high-fidelity capture of expressive articulation on period instruments whose actions cannot be modified. The explicit open-source release of complete hardware and firmware is a concrete strength that directly lowers barriers to replication and extension, potentially supporting the construction of instrument-specific MIDI corpora grounded in real historical mechanisms.

major comments (1)
  1. [Abstract] Abstract and system description: the central claims of 'high-resolution, low-latency sensing' and reliable operation 'across multiple manuals and variable key counts' are presented without any quantitative performance data, error analysis, resolution measurements, latency benchmarks, or validation experiments against ground truth. This absence is load-bearing because the weakest assumption (that low-profile reflective sensors can deliver the stated performance without instrument-specific calibration or playability changes) cannot be evaluated from the design description alone.
minor comments (1)
  1. The abstract and introduction would benefit from a short table or bullet list summarizing key specifications (e.g., sensor model, nominal resolution, maximum key count per manual) to allow readers to assess feasibility immediately.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive feedback and for acknowledging PHOTON's potential to enable non-invasive capture of expressive articulation on historical instruments. We address the single major comment below and have revised the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract and system description: the central claims of 'high-resolution, low-latency sensing' and reliable operation 'across multiple manuals and variable key counts' are presented without any quantitative performance data, error analysis, resolution measurements, latency benchmarks, or validation experiments against ground truth. This absence is load-bearing because the weakest assumption (that low-profile reflective sensors can deliver the stated performance without instrument-specific calibration or playability changes) cannot be evaluated from the design description alone.

    Authors: We agree that the abstract and system description would be strengthened by quantitative evidence. The submitted manuscript prioritizes the novel low-profile, modular design and complete open-source release (KiCad hardware and CircuitPython firmware) to lower barriers for replication on diverse historical instruments. To directly address the concern, the revised version will add a new 'System Characterization' section containing: resolution and accuracy measurements using a calibrated linear displacement stage as ground truth; end-to-end latency benchmarks from sensor output to MIDI event; error analysis across key velocities, multiple manuals, and variable key counts; and confirmation via performer testing that mounting produces no measurable change in playability or action feel. We will also document the minimal calibration procedure and its instrument independence. These additions will allow readers to evaluate the performance claims objectively. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is a pure engineering system description of a modular optical sensing architecture for historical keyboards. It contains no mathematical derivations, equations, predictions, fitted parameters, or uniqueness theorems. The central claims follow directly from the stated non-contact, low-profile design choices and are supported by the open-source release rather than any internal reduction to inputs. No self-citation chains or ansatzes are invoked to justify core results.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard engineering assumptions about optical sensing accuracy and mechanical compatibility rather than new postulates or fitted parameters.

axioms (1)
  • domain assumption Reflective optical sensors can provide accurate continuous displacement measurements of key levers without mechanical contact or interference.
    Invoked to support the non-invasive and high-resolution claims.

pith-pipeline@v0.9.0 · 5506 in / 1340 out tokens · 49065 ms · 2026-05-08T13:11:15.289421+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 9 canonical work pages

  1. [1]

    Alf Gabrielsson. 2003. Music Performance Research at the Millennium.Psychol- ogy of Music31, 3 (2003), 221–272. https://doi.org/10.1177/03057356030313003

    work page doi:10.1177/03057356030313003 2003

  2. [2]

    Bruno Gingras, Pierre-Yves Asselin, Meghan Goodchild, and Stephen McAdams. 2009. Communicating Voice Emphasis in Harpsichord Perfor- mance. InProceedings of the 7th Triennial Conference of the European Society for the Cognitive Sciences of Music (ESCOM 2009), Jukka Louhivuori, Tuo- mas Eerola, Suvi Saarikallio, Tommi Himberg, and Päivi-Sisko Eerola (Eds.)...

    2009

  3. [3]

    Bruno Gingras, Pierre-Yves Asselin, and Stephen McAdams. 2013. Individuality in Harpsichord Performance: Disentangling Performer- and Piece-specific Influences on Interpretive Choices.Frontiers in Psychology4 (2013), 895. https://doi.org/10.3389/fpsyg.2013.00895

    work page doi:10.3389/fpsyg.2013.00895 2013

  4. [4]

    Matthew Hamilton, Michele Ducceschi, Roberto Livi, Catalina Vicens, and An- drew McPherson. 2025. Augmentation of a Historical Harpsichord Keyboard Replica for Haptic-Enabled Interaction in Museum Exhibitions. InProceedings of the 2025 International Conference on New Interfaces for Musical Expression (NIME). https://nime.org/proceedings/2025/nime2025_61.pdf

    2025

  5. [5]

    Randall Harlow, Mattias Petersson, Robert Ek, Federico Visi, and Stefan Öster- sjö. 2021. Global Hyperorgan: A Platform for Telematic Musicking and Re- search. InProceedings of the International Conference on New Interfaces for Musical Expression (NIME). https://doi.org/10.21428/92fbeb44.d4146b2d Pub- lished Apr 29, 2021

    work page doi:10.21428/92fbeb44.d4146b2d 2021

  6. [6]

    Kaori Kuromiya, Yuya Kobayashi, Masato Hirano, and Shinichi Furuya. 2025. Motor Origins of Timbre in Piano Performance.Proceedings of the National Academy of Sciences122, e2425073122 (2025). https://doi.org/10.1073/pnas. 2425073122

    work page doi:10.1073/pnas 2025

  7. [7]

    Tod Machover and Joe Chung. 1989. Hyperinstruments: Musically Intelligent and Interactive Performance and Creativity Systems. InProceedings of the International Computer Music Conference (ICMC). 186–190. http://hdl.handle. net/2027/spo.bbp2372.1989.046

    work page arXiv 1989

  8. [8]

    Jennifer MacRitchie and Giulia Nuti. 2015. Using Historical Accounts of Harpsichord Touch to Empirically Investigate the Production and Perception of Dynamics on the 1788 Taskin.Frontiers in Psychology6 (2015), 183. https: //doi.org/10.3389/fpsyg.2015.00183

    work page doi:10.3389/fpsyg.2015.00183 2015

  9. [9]

    McPherson

    Andrew P. McPherson. 2013. Portable Measurement and Mapping of Con- tinuous Piano Gesture. InProceedings of the International Conference on New Interfaces for Musical Expression (NIME 2013). Daejeon, South Korea, 152–157

    2013

  10. [10]

    McPherson and Youngmoo E

    Andrew P. McPherson and Youngmoo E. Kim. 2011. Multidimensional Gesture Sensing at the Piano Keyboard. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). Vancouver, Canada, 2789–2798. https://doi.org/10.1145/1978942.1979355

    work page doi:10.1145/1978942.1979355 2011

  11. [11]

    Will Mowat. 2005. Bob Moog Piano Bar.Sound on Sound(2005). https: //www.soundonsound.com/reviews/bob-moog-piano-bar Online review

    2005

  12. [12]

    Takanori Oku and Shinichi Furuya. 2022. Noncontact and High-Precision Sensing System for Piano Keys Identified Fingerprints of Virtuosity.Sensors 22 (2022). https://doi.org/10.3390/s22134891

    work page doi:10.3390/s22134891 2022

  13. [13]

    Henri Penttinen. 2006. On the Dynamics of the Harpsichord and Its Syn- thesis. InProceedings of the 9th International Conference on Digital Audio Effects (DAFx). Montreal, Canada. https://aaltodoc.aalto.fi/server/api/core/ bitstreams/df2646b3-5b94-4e0d-9719-01a19f941c8a/content

    2006

  14. [14]

    QRS Music Technologies. 2024. Roll with the Punches: The Story of QRS Music. http://www.qrsmusic.com/assets/pdf/BKS24.pdf Company self-publication on the history of QRS and player piano technology

    2024

  15. [15]

    Adam Schmidt, Jeffrey Snyder, Gian Torrano Jacobs, Joseph Gascho, Joyce Chen, and Andrew McPherson. 2025. The Sparksichord: Practical Implemen- tation of a Lorentz Force Electromagnetic Actuation and Feedback System. In Proceedings of the 2025 International Conference on New Interfaces for Musical Expression (NIME). Canberra, Australia. https://nime.org/p...

    2025

  16. [16]

    Adam Štefunko, Carlos Eduardo Cancino-Chacón, and Jan Jr. Hajič. 2026. On Every Note a Griff: Looking for a Useful Representation of Basso Continuo Performance Style. arXiv:2601.20478 [cs.SD] https://arxiv.org/abs/2601.20478

    work page arXiv 2026

  17. [17]

    Adam Štefunko, Suhit Chiruthapudi, Jan Hajic, and Carlos Eduardo Cancino- Chacón. 2025. Basso Continuo Goes Digital: Collecting and Aligning a Sym- bolic Dataset of Continuo Performance. InProceedings of the 6th Conference on AI Music Creativity (AIMC). Brussels, Belgium

    2025