$J/\psi$ production within a jet in high-energy proton-proton and nucleus-nucleus collisions

Within the framework of leading power factorization formalism of nonrelativistic quantum chromodynamics, we calculate the jet fragmentation function for $J/\psi$ production in proton-proton (pp) collisions ranging from $\sqrt{s}=500$ GeV to $13$ TeV. The reasonable agreements between theory and experimental data indicate that $J/\psi$ production within a jet is mainly dominated by gluon fragmentation. Such a mechanism can be further tested by the predicted jet transverse momentum and radius dependence of jet fragmentation function. Based on the satisfying description of pp baseline, we carry out the first theoretical investigation on medium modification on $J/\psi$ production within jet in heavy-ion collisions at the Large Hadron Collider, using a linear Boltzmann transport model combined with hydrodynamics for the simulation of jet-medium interaction. The consistency with the experimental measurement on nuclear modification factor $R_{\text{AA}}$ by CMS collaboration reveals that the gluon jet quenching is the driving force for the suppression of $J/\psi$ production in jet. Furthermore, we make predictions for the dependence of $R_{\text{AA}}$ on the jet transverse momentum and jet radius $R$, which can be tested in future measurements to further constrain the flavor dependence of jet quenching.