Pith. sign in

REVIEW 5 major objections 5 minor 33 references

Reward-Augmented Reinforcement Learning for Continuous Control in Precision Autonomous Parking via Policy Optimization Methods

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Milestone-augmented reward, combined with on-policy optimization, lifts continuous-steering parking success to 91% in a custom 3D simulator.

desk verdict Useful reward-shaping comparison for continuous parking control, but the 91% success claim is built on a trigger-based success criterion that is not precision parking. read the letter →

arxiv 2507.19642 v2 pith:QIQA66CN submitted 2025-07-25 cs.RO cs.SYeess.SY

Ahmad Suleman , Misha Urooj Khan , Zeeshan Kaleem , Ali H. Alenezi , Iqra Shabbir , Sinem Coleri , Chau Yuen This is my paper · ORCID
classification cs.ROcs.SYeess.SY
keywords autonomousparkingrewardaugmentationcontinuouscontroldeepreinforcementlearningon-policyoptimizationoff-policyUnity3Dsimulationmilestone
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the design of the reward signal, rather than the choice of policy-optimization algorithm, is the decisive factor in learning continuous steering for autonomous parking. It builds a custom 3D parking simulator with a model-free Markov decision process, continuous steering actions, and raycast sensing, and compares three reward strategies: goal-only, dense proximity, and milestone-augmented. The reported result is that milestone-augmented reward combined with on-policy optimization reaches a 91% success rate with smooth trajectories, while goal-only and dense proximity rewards fail. All numbers are reported within the paper's own environment, which the paper notes does not permit direct comparison against standardized benchmarks. If true, the finding gives practitioners a concrete recipe: insert a single spatially grounded milestone into the reward to make continuous-domain parking learnable.

What carries the argument

The load-bearing object is the milestone-augmented reward function $R^{\mathrm{MAR}}_t$, a semi-dense reward that switches from a small living penalty to a proximity term $\zeta - D(x_a,x_g)$ once the indicator $\mathrm{Ind}_M = \mathbb{I}[D(x_a,x_m) \le M^{\mathrm{XYZ}}_{\mathrm{mil}}]$ reports entry into a hand-placed milestone region. This single switch converts an almost-sparse signal into one that gives mid-trajectory guidance, and the paper attributes the 91% success rate to it. The switch is embedded in an on-policy clipped surrogate objective with generalized advantage estimation, a Gaussian policy over a scalar steering command in $[-1,1]$, and twelve parallel simulation environments sharing one policy network.

What would settle it

Replay the trained ON-POM-MAR policy and record the vehicle pose at the instant the arrival collider fires. If the distribution of $\|x_a - x_g\|_2$ has a median well above the parking slot depth, or the yaw error is more than a few degrees in a large fraction of episodes, then the 91% success rate measures arrival-trigger proximity, not precision parking.

Watch

Extended reading notes

Core claim

The central claim is that RARLAP, a reward-augmented reinforcement-learning framework, makes precision parking learnable in a continuous-action setting by structuring the reward rather than by adding demonstrations, planners, or modular decomposition. The paper argues that a sparse goal-only reward starves the agent of feedback, and a dense proximity reward drives it into local oscillatory behavior, whereas a milestone-augmented reward that switches from a living penalty to proximity feedback once the vehicle enters a defined region produces stable learning. Under that reward, on-policy optimization is reported to achieve a 91% success rate, a 9% collision rate, and an average final distance of 2.191 m, substantially outperforming the off-policy variant, which reaches 50% success and 7.287 m. The paper concludes that milestone-based reward augmentation is a fundamental driver of learning stability and behavioral quality, and that on-policy updates exploit that structure better than off-policy replay.

Load-bearing premise

The load-bearing premise is that the simulator's transparent collider at the parking destination defines a successful precision park, even though successful episodes end on average 2.191 m from the goal; if that trigger fires before the vehicle is properly aligned inside the slot, both the 91% success rate and the precision claim collapse.

Editorial extensions

If this is right

  • Reward structure, not just the choice between on-policy and off-policy learning, determines whether continuous steering policies converge to safe parking behavior.
  • On-policy optimization with MAR is reported to reach 91% success, 9% collision, and 2.191 m average final distance, while off-policy MAR reaches 50% success and 7.287 m, implying that synchronous policy updates exploit the milestone signal better than replay-based updates.
  • The failure of both GOR and DPR indicates that sparse rewards starve the agent and poorly structured dense rewards destabilize it; semi-dense milestone shaping is presented as the middle path.
  • The released simulator with parallel environments and matched hyperparameters provides a controlled testbed for comparing reward design and policy optimization paradigms.
  • The smooth, goal-directed trajectories under MAR suggest the learned policy behaves as a potential-field controller, which the paper argues is better suited to obstacle-rich parking scenarios.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves open is to sweep the milestone radius $M^{\mathrm{XYZ}}_{\mathrm{mil}}$: the 91% success rate is reported for a single placement, and the optimal radius may depend on the parking slot geometry.
  • An alternative interpretation of the OFF-POM shortfall is that replay buffers dilute the milestone signal; pairing OFF-POM with hindsight relabeling or prioritized replay on the same MAR reward would separate reward structure from algorithm choice.
  • The paper itself notes that comparisons against standardized benchmarks were not applicable, so the headline numbers are meaningful as intra-environment comparisons; transferring the method to a public parking benchmark would test its generality.
  • The MAR switch from a living penalty to proximity feedback is a form of reward shaping; if it is equivalent to a potential-based shaping function, the same 91% behavior could transfer to other goal-reaching tasks without changing the algorithm.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes RARLAP, a reward-augmented reinforcement learning framework for continuous-domain steering control in an autonomous parking task. Three reward strategies—sparse goal-only reward (GOR), dense proximity reward (DPR), and milestone-augmented reward (MAR)—are defined and evaluated under both an on-policy optimization mechanism (ON-POM) and an off-policy one (OFF-POM) in a custom Unity 3D simulator. The central claim is that ON-POM trained with MAR achieves a 91% success rate, smoother trajectories, and more robust behavior, while GOR and DPR fail to guide effective learning. The paper also reports convergence and trajectory analyses and provides open-source code.

Significance. If the central claim survived scrutiny, the paper would be a useful systematic comparison of reward-structure design for continuous-control parking, with a reproducible open-source simulator as a practical contribution. The question of how reward shaping interacts with on-policy versus off-policy optimization is relevant to a broad RL-for-robotics audience. However, the current evaluation does not establish the claim: the success criterion is a collider-triggered event rather than a geometric parking condition, the statistical tests are applied to autocorrelated samples without proper replication, and key metrics such as average final distance are inconsistent with the precision-parking framing. These problems are load-bearing for the paper's headline conclusions.

major comments (5)
  1. The success definition used throughout the paper is not a parking-completion criterion. Section IV-A3 states that small transparent colliders at parking destinations trigger arrival events, so the 91% success rate in Table IV likely records overlap with a collider volume rather than the vehicle being correctly aligned inside the parking slot. This interpretation is supported by the reported average final distance of 2.191 m for ON-POM(MAR) on a vehicle that is 4 m long; a truly parked vehicle should have near-zero final distance from the slot reference. The paper provides no per-episode pose error, heading error, or slot-containment metric. Because the abstract's 'precision parking' claim and the entire MAR versus GOR/DPR comparison rest on this binary success flag, the central result is not validated. Please redefine success using a geometric criterion (for example, all four wheels inside the slot and heading within a tolerance), recompute success/collision rates, and report pose-error metrics for both successful and failed episodes.
  2. The statistical significance analysis is not valid as presented. The Welch t-tests are applied to 1000 reward values sampled every 2000 steps during training; these samples are temporally autocorrelated within a single run and are not independent observations. Furthermore, no number of independent seeds is reported, and Table IV provides no standard deviations, confidence intervals, or per-seed breakdown for the success rates, collision rates, or average final distance. The reported t-values and Cohen's d therefore do not substantiate the claim that MAR is significantly better than GOR or DPR. Please run multiple seeds (at least 5–10) per condition, report mean±std across seeds, and use statistical tests that avoid pseudoreplication, such as testing on per-seed aggregate scores rather than on individual time steps.
  3. The 'Avg. Final Distance' metric is undefined and internally inconsistent with the success rate. For ON-POM(MAR), a 91% success rate coexists with an average final distance of 2.191 m, which suggests that either the distance is measured from an arbitrary reference (for example, from the vehicle's initial position or from a point far outside the slot) or the 'successful' episodes do not end with the vehicle parked. The paper does not state what points are used in the distance computation, nor does it separate final distance for successful versus failed episodes. This makes the precision claim untestable. Please define the metric precisely, report final-distance/heading error conditional on success, and reconcile the success rate with a final distance of a few centimeters for genuinely parked vehicles.
  4. The reward functions contain multiple free parameters—F_env^G, F_env^C, F_env^L in (6), p_alpha and p_beta in (7), zeta and M_mil^XYZ in (8)–(9), and the milestone position x_m—but no sensitivity analysis or ablation is provided. The paper's conclusion is that the milestone structure itself, not the particular numerical coefficients, is responsible for MAR's superiority. Without a coarse sweep over reward magnitudes, the milestone radius, and the milestone location, one cannot rule out that the observed ranking is an artifact of favorable parameter choices for MAR and unfavorable choices for DPR. Even a limited parameter sweep would substantially strengthen the claim that the reward structure, rather than tuning, drives the result.
  5. The 'smoother trajectories' and 'more robust behavior' claims are supported only by qualitative plots of selected trajectories. No quantitative smoothness metric (for example, steering-rate variance, path curvature, or jerk) is reported, and the plots appear to show single episodes rather than distributions over many rollouts. Please provide quantitative smoothness and robustness measures computed over multiple seeds and episodes, with confidence intervals, so that the behavioral claims in the abstract and Section IV can be evaluated.
minor comments (5)
  1. Equation (15) defines the composite loss for ON-POM but labels it 'L_off-policy'; the subscript should be 'L_on-policy' or 'L_composite' to match the on-policy context.
  2. The header and the row for the proposed work contain typos: 'Propoed' should be 'Proposed', and 'Propoed RARLAPOn-policy' should be separated into readable text.
  3. Some references appear mismatched to their citation context: [5] is about charging infrastructure, not model-predictive control as cited in Section I-A, and [9] is about path loss models, not reward feedback as cited in Section I. Please verify and correct the reference list.
  4. The text states that the shaping parameter zeta ensures the reward remains non-negative after the milestone is passed, but the term zeta - D(x_a, x_g) is negative whenever D(x_a, x_g) exceeds zeta; no constraint guaranteeing D <= zeta is stated. Please clarify or impose an explicit bound.
  5. The comparison of training times (26.83 min vs. 268.74 min) is presented without reporting the hardware configuration, the number of parallel environments, or whether the times include evaluation; this makes the efficiency claim difficult to interpret or reproduce.

Circularity Check

1 steps flagged · score 4.0 of 10

MAR-versus-GOR/DPR superiority is partly circular because the main statistical comparison uses reward values whose magnitudes MAR increases by construction; the absolute success metrics are independent but secondary.

  1. other [Section IV-B (Evaluation of Reward-Based Learning Performance), first paragraph; reward definitions in Eqs. (6), (7), (9).]
    "We begin by assessing the statistical significance of performance differences using independent two-sample Welch’s t-tests on 1000 reward values sampled every 2000 steps during training. ... The results strongly support the superiority of the MAR strategy. In the on-policy setting, ON-POM–MAR significantly outperforms both ON-POM–GOR (t= 53.46, p≪0.001, Cohen’s d= 2.39) and ON-POM–DPR (t= 70.32, p≪0.001, Cohen’s d= 3.14)"

    MAR (Eq. 9) pays an extra ζ−D(x_a,x_g) whenever IndM=1, while GOR (Eq. 6) pays only the small living penalty −F_L in the same otherwise-branch. For identical behavior that crosses the milestone, MAR therefore collects larger cumulative reward than GOR or DPR by construction. The paper's headline statistical evidence is a t-test on those reward values, so the 'MAR significantly outperforms' result is partly a restatement of the reward definition rather than an independent measure of parking quality; success/collision statistics for GOR and DPR are not reported.

full rationale

RARLAP is an empirical simulation study rather than a chain of analytic derivations, so most derivation-based circularity patterns do not apply. No load-bearing self-citations or imported uniqueness theorems are used; the references are standard external works. The strongest numerical claims (Table IV: 91% success, 9% collisions, 2.191 m final distance for ON-POM MAR) are reported measurements and were not obtained by fitting the reward to that metric. The one genuinely circular evidential step is the statistical comparison of MAR against GOR/DPR: it is computed on cumulative reward values whose magnitudes MAR inflates by design through its milestone bonus. That makes the comparative claim partly self-definitional, although the absolute success rate and the distance-to-goal/trajectory plots retain some independent content. The separate issue that success is triggered by transparent colliders rather than geometric slot alignment is better characterized as a validity or correctness risk, not as circularity of a derivation, so it is noted but does not increase the circularity score.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The central claim rests on hand-designed reward constants, a hand-placed milestone, and a simulator-defined success trigger. None of these are benchmarked against external data, so the reported 91% success is a within-environment result rather than a general parking result.

free parameters (4)
  • Reward magnitudes F_env^G, F_env^C, F_env^L = not reported
    Hand-chosen scalar rewards for goal, collision, and living penalties in Eqs. 6, 7, and 9; values are never listed and directly determine which reward strategy wins.
  • DPR scaling parameters p_alpha, p_beta = not reported
    Eq. 7's dense proximity reward depends on these positive scaling constants; they set the reward slope that the paper claims DPR gets wrong.
  • MAR shaping parameter zeta and milestone radius M_mil^XYZ = not reported
    Eq. 9 needs zeta to keep post-milestone reward non-negative and Eq. 8 needs the milestone radius; neither is specified.
  • Milestone placement x_m = not reported
    The key augmentation is a manually placed midway region; its coordinates are omitted, so the headline method cannot be reproduced exactly from the text.
assumptions (3)
  • domain assumption The Unity simulation with raycast sensors and collider-based arrival detection is a faithful proxy for real parking performance.
    All conclusions about precision and safety rest on simulator fidelity; the paper provides no sim-to-real validation (Section IV-A).
  • domain assumption Matched hyperparameters across PPO and SAC are a sufficient control for a fair algorithm comparison.
    Section IV-A5 claims fairness from identical batch size, learning rate, discount factor, and network sizes, but neither algorithm is tuned, which may favor or penalize one paradigm.
  • standard math Reward shaping with a milestone does not change the optimal policy, only learning behavior.
    This is the usual reward-shaping intuition in RL, but the paper never invokes a potential-based shaping guarantee, and its own DPR result shows shaping can change final outcomes.
invented entities (1)
  • Milestone region M_mil^XYZ
    purpose: A spatially grounded reward checkpoint meant to guide the agent through a favorable region before the parking slot.
    Introduced in Eq. 8 and Fig. 3(c); there is no independent evidence that this specific region improves real parking performance, and its coordinates are not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reward-Augmented Reinforcement Learning for Continuous Control in Precision Autonomous Parking via Policy Optimization Methods." pith.science (2026). https://pith.science/paper/QIQA66CN

@misc{pith2026250719642,
  author       = {Pith},
  title        = {Pith review of: Reward-Augmented Reinforcement Learning for Continuous Control in Precision Autonomous Parking via Policy Optimization Methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QIQA66CN}},
  note         = {Machine review of arXiv:2507.19642}
}
read the original abstract

Autonomous parking (AP) represents a critical yet complex subset of intelligent vehicle automation, characterized by tight spatial constraints, frequent close-range obstacle interactions, and stringent safety margins. However, conventional rule-based and model-predictive methods often lack the adaptability and generalization needed to handle the nonlinear and environment-dependent complexities of AP. To address these limitations, we propose a reward-augmented learning framework for AP (RARLAP), that mitigates the inherent complexities of continuous-domain control by leveraging structured reward design to induce smooth and adaptable policy behavior, trained entirely within a high-fidelity Unity-based custom 3D simulation environment. We systematically design and assess three structured reward strategies: goal-only reward (GOR), dense proximity reward (DPR), and milestone-augmented reward (MAR), each integrated with both on-policy and off-policy optimization paradigms. Empirical evaluations demonstrate that the on-policy MAR achieves a 91\% success rate, yielding smoother trajectories and more robust behavior, while GOR and DPR fail to guide effective learning. Convergence and trajectory analyses demonstrate that the proposed framework enhances policy adaptability, accelerates training, and improves safety in continuous control. Overall, RARLAP establishes that reward augmentation effectively addresses complex autonomous parking challenges, enabling scalable and efficient policy optimization with both on- and off-policy methods. To support reproducibility, the code accompanying this paper is publicly available.

Figures

Figures reproduced from arXiv: 2507.19642 by the authors.

Figure 1
Figure 1. Unity objects with actual meshes and colliders used in environment design (a) Vehicle body mesh (b) Obstacle-vehicle’s [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Markov decision process environment model of au [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. (a) Environment layout with driving agent, occupied and empty parking slots (b) Agent’s action space control (c) [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Illustration of parallel environment rollouts used to [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: ON-POM and OFF-POM Training w.r.t (a) GOR (b) DPR and (c) MAR based feedback functions, policy and value [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison across reward strategies for best performing ON-POM AP-agent: (a–c) Euclidean distance to terminal goal [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Policy smoothness comparison through learned trajec [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 32 canonical work pages

  1. [1]

    Diffusion Models for Autonomous Driving in Smart Parking: A Data Synthesis Frame- work,

    M. Zhou, X. Zhang, J. Yin, Y . Hu, W. Liu and J. Li, “Diffusion Models for Autonomous Driving in Smart Parking: A Data Synthesis Frame- work,”IEEE Transactions on Automation Science and Engineering, 2025

    work page 2025

  2. [2]

    Exploiting multi-modal fusion for ur- ban autonomous driving using latent deep reinforcement learning,

    Y . H. Khalil and H. T. Mouftah, “Exploiting multi-modal fusion for ur- ban autonomous driving using latent deep reinforcement learning,”IEEE Transactions on Vehicular Technology, vol. 72, no. 3, pp. 2921–2935, 2022

    work page 2022

  3. [3]

    An Anytime Trajectory Optimizer for Accurately Parking an Autonomous Vehicle in Tiny Spaces,

    X. Chen, Y . Sun, T. Zhang, X. Wang, S. Xiong and K. Cao, “An Anytime Trajectory Optimizer for Accurately Parking an Autonomous Vehicle in Tiny Spaces,”IEEE Transactions on Vehicular Technology, vol. 74, no. 6, pp. 8772-8783, 2025

    work page 2025

  4. [4]

    The autonomous vehicle parking problem,

    A. Millard-Ball, “The autonomous vehicle parking problem,”Transport Policy, vol. 75, pp. 99–108, 2019

    work page 2019

  5. [5]

    Optimal charging of plug-in electric vehicles for a car-park infrastructure,

    T. Ma and O. A. Mohammed, “Optimal charging of plug-in electric vehicles for a car-park infrastructure,”IEEE Transactions on Industry Applications, vol. 50, no. 4, pp. 2323–2330, 2014. 11

    work page 2014

  6. [6]

    Driving tasks transfer using deep reinforcement learning for decision-making of autonomous vehicles in unsignalized intersection,

    H. Shu, T. Liu, X. Mu, and D. Cao, “Driving tasks transfer using deep reinforcement learning for decision-making of autonomous vehicles in unsignalized intersection,”IEEE Transactions on Vehicular Technology, vol. 71, no. 1, pp. 41–52, 2021

    work page 2021

  7. [7]

    Deep reinforcement learning for solving vehicle routing problems with backhauls,

    C. Wang, Z. Cao, Y . Wu, L. Teng, and G. Wu, “Deep reinforcement learning for solving vehicle routing problems with backhauls,”IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 3, pp. 4779-4793, March 2025

    work page 2025

  8. [8]

    A hybrid deep reinforcement learning for autonomous vehicles smart- platooning,

    S. B. Prathiba, G. Raja, K. Dev, N. Kumar, and M. Guizani, “A hybrid deep reinforcement learning for autonomous vehicles smart- platooning,”IEEE Transactions on Vehicular Technology, vol. 70, no. 12, pp. 13340–13350, 2021

    work page 2021

Show all 33 references
  1. [9]

    Path loss models for low-power, low-data rate sensor nodes for smart car parking systems,

    T. O. Olasupo, C. E. Otero, L. D. Otero, K. O. Olasupo, and I. Kostanic, “Path loss models for low-power, low-data rate sensor nodes for smart car parking systems,”IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 6, pp. 1774–1783, 2017

    2017

  2. [10]

    Learning task-relevant representations via rewards and real actions,

    L. Yuanet al., “Learning task-relevant representations via rewards and real actions,”Knowledge-based systems, vol. 294, p. 111788, 2024

    2024

  3. [11]

    Comprehensive overview of reward engineering and shaping,

    S. Ibrahimet al., “Comprehensive overview of reward engineering and shaping,”IEEE Access, 2024

    2024

  4. [12]

    Majumder,Deep Reinforcement Learning in Unity, Springer, 2020

    A. Majumder,Deep Reinforcement Learning in Unity, Springer, 2020

    2020

  5. [13]

    Vision-based autonomous driving: A hierarchical reinforcement learning approach,

    J. Wang, H. Sun, and C. Zhu, “Vision-based autonomous driving: A hierarchical reinforcement learning approach,”IEEE Transactions on Vehicular Technology, vol. 72, no. 9, pp. 11213–11226, 2023

    2023

  6. [14]

    Unity: A general platform for intelligent agents,

    A. Juliani, V .-P. Berges, E. Teng, A. Cohen, J. Harper, C. Elion, C. Goy, Y . Gao, H. Henry, M. Mattar, and D. Lange, “Unity: A general platform for intelligent agents,”arXiv preprint arXiv:1809.02627, 2020

    2020 arXiv

  7. [15]

    A survey of autonomous driving: Common practices and emerging technologies,

    E. Yurtseveret al., “A survey of autonomous driving: Common practices and emerging technologies,”IEEE Access, vol. 8, pp. 58443–58469, 2020

    2020

  8. [16]

    Deep learning-based perception systems for autonomous driving,

    L.-H. Wen and K.-H. Jo, “Deep learning-based perception systems for autonomous driving,”Neurocomputing, vol. 489, pp. 255–270, 2022

    2022

  9. [17]

    Nonlinear MPC on Parallel Parking for Autonomous Vehicles Under State-Dependent Switching,

    H. Zhao, H. Yang, Z. Wang and Y . Xia, “Nonlinear MPC on Parallel Parking for Autonomous Vehicles Under State-Dependent Switching,” IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 6377-6387, 2025

    2025

  10. [18]

    Robust Real-Time Sampling-Based Motion Planner for Autonomous Vehicles in Narrow Environments,

    M. Kim, A. Esquerre-Pourt `ere and J. Park, “Robust Real-Time Sampling-Based Motion Planner for Autonomous Vehicles in Narrow Environments,”IEEE Transactions on Automation Science and Engi- neering, vol. 22, pp. 16250-16265, 2025

    2025

  11. [19]

    Simulated autonomous driving using reinforcement learning: Unity ML-agents,

    Y . Savidet al., “Simulated autonomous driving using reinforcement learning: Unity ML-agents,”Information, vol. 14, no. 5, p. 290, 2023

    2023

  12. [20]

    Autonomous car parking using deep reinforcement learning,

    R. Takehara and T. Gonsalves, “Autonomous car parking using deep reinforcement learning,” inProceedings of the 2021 International Con- ference on Innovative Technology (ICITech), Kuala Lumpur, Malaysia, 2021, pp. 85–89

    2021

  13. [21]

    Hierarchical reinforcement learning for autonomous parking,

    X. Xu, Y . Li, Z. Zhang, and M. Chen, “Hierarchical reinforcement learning for autonomous parking,” inProceedings of the 2024 IEEE In- ternational Conference on Robotics and Biomimetics (ROBIO), Beijing, China, 2024

    2024

  14. [22]

    Autonomous valet parking with A3C and PPO,

    T. Tiong, K. H. Chan, Y . Quek, and H. Lim, “Autonomous valet parking with A3C and PPO,” inProceedings of the 2022 IEEE Consumer Communications & Networking Conference (CCNC), Las Vegas, NV , USA, 2022, pp. 334–340

    2022

  15. [23]

    DQN implementation for simulated autonomous vehicle,

    Y . Quek, K. H. Chan, and M. H. Ang, “DQN implementation for simulated autonomous vehicle,”IET Intelligent Transport Systems, vol. 15, no. 6, pp. 679–687, 2021

    2021

  16. [24]

    Comparative analysis of loss functions in TD3,

    K. H. Chan, Y . Quek, and M. H. Ang, “Comparative analysis of loss functions in TD3,”Journal of Soft Computing and Data Mining, vol. 5, no. 1, pp. 1–10, 2024

    2024

  17. [25]

    Vision-guided parallel parking system for a mobile robot,

    M. Shaker, D. Mikic, and A. Skrjanc, “Vision-guided parallel parking system for a mobile robot,” inProceedings of the 2010 IEEE Interna- tional Conference on Industrial Technology (ICIT), Vina del Mar, Chile, 2010, pp. 921–926

    2010

  18. [26]

    Double bootstrapped SAC- discrete for autonomous vehicles,

    J. Yang, Z. Gu, X. Wu, and Y . Xu, “Double bootstrapped SAC- discrete for autonomous vehicles,”IEEE Transactions on Cognitive and Developmental Systems, vol. 15, no. 4, pp. 2041–2052, 2021

    2021

  19. [27]

    Improved reinforcement learn- ing via knowledge transfer for traffic signal control,

    D. Ding, L. Xu, Y . Wang, and Y . Chen, “Improved reinforcement learn- ing via knowledge transfer for traffic signal control,”Neurocomputing, vol. 365, pp. 281–293, 2019

    2019

  20. [28]

    DRLeague: A 3D rein- forcement learning environment for simulating robotic soccer agents,

    H. Farrapo, M. Mendonc ¸a, and A. L. Braga, “DRLeague: A 3D rein- forcement learning environment for simulating robotic soccer agents,” inProc. SBGames, Recife, Brazil, 2022, pp. 134–143

    2022

  21. [29]

    D’Alfonso, F

    L. D’Alfonso, F. Giannini, G. Franz `e, G. Fedele, F. Pupo, and G. Fortino, ”Autonomous vehicle platoons in urban road networks: A joint distributed reinforcement learning and model predictive control approach,”IEEE/CAA Journal of Automatica Sinica, vol. 11, no. 1, pp. 141–156, 2024

    2024

  22. [30]

    A review of reward functions for reinforcement learning in the context of autonomous driving,

    A. Abouelazm, J. Michel, and J. M. Z ¨ollner, “A review of reward functions for reinforcement learning in the context of autonomous driving,” in2024 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2024, pp. 156–163

    2024

  23. [31]

    Dayal, L

    A. Dayal, L. R. Cenkeramaddi, and A. Jha, ”Reward criteria impact on the performance of reinforcement learning agent for autonomous navigation,”Applied Soft Computing, vol. 126, p. 109241, 2022

    2022

  24. [32]

    Unity Asset Store

    Unity Technologies, “Unity Asset Store.” [Online]. Available: https://assetstore.unity.com/ Ahmad Sulemanis currently serving in the National Centre for Physics (NCP) as Assistant Manager and as vice chairperson in the community of research and development (CRD). He holds an M...

    2007

  25. [2018]

    He won the Higher Education Commission (HEC) Best Innovator Award in 2017, with a single award from all over Pakistan

    We won the Runner-up Award in the National Hackathon 23 competition for Project to develop Drone Detection system. He won the Higher Education Commission (HEC) Best Innovator Award in 2017, with a single award from all over Pakistan. He received the 2021 Top Reviewer Recogniti...

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.