Pith. sign in

REVIEW 3 major objections 6 minor 57 references

LineFlow: A Framework to Learn Active Control of Production Lines

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LineFlow introduces a standardized RL benchmark for production-line control, using provable optima as the yardstick for what learning achieves.

desk verdict A genuinely useful open-source RL benchmark for production-line control, with one overclaimed 'optimal' baseline in the non-stationary WTJ scenario. read the letter →

arxiv 2505.06744 v1 pith:27UDU4UO submitted 2025-05-10 cs.LG stat.ML

Kai Müller , Martin Wenzel , Tobias Windisch This is my paper · ORCID
classification cs.LGstat.ML MSC 68T0590B30
keywords activelinecontrolreinforcementlearningdiscrete-eventsimulationproductionwaitingtimeoptimizationworkerassignmentpartdistributioncurriculum
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LineFlow is presented as the missing standardized piece of the production-control RL pipeline: an extensible, open-source simulator that lets researchers build production lines of arbitrary layout and train agents to control them. To make the question "did the agent learn the right thing" answerable, the authors formulate four core subproblems—source waiting time with and without jumps, part distribution over parallel stations, and worker assignment—and derive closed-form or exact optima for each. Their benchmark results show that standard RL algorithms approach these theoretical optima on the simple subproblems, with recurrent policies doing best on the non-stationary waiting-time scenario. On a combined "complex line" that requires memory, planning, and scrap avoidance, vanilla policies deadlock at zero reward, curriculum learning recovers positive reward, yet the best learned policy still falls short of a manually designed heuristic. The point of the paper is that such benchmarks make RL progress in active line control measurable, and the hard scenarios show where the open problems are.

What carries the argument

The central object is LineFlow itself: a discrete-event simulator in which stations (sources, processes, assemblies, sinks, switches, worker pools) are connected by finite-capacity FIFO buffers and every processing time has the form $T + \mathrm{Exp}_S$ (a minimum plus an exponential delay). The measuring instrument is the accumulated value $C^\pi(t) = T_C/t\,(c\, n^\pi_{\mathrm{ok}}(t) - \sum_i c_i n^\pi_{\mathrm{nok}}(t,i))$, which converts produced and scrapped parts into reward. The load-bearing machinery is the set of provable optima: the static waiting time $T^*_W = \mathbb{E}[T_A + 2T_g - T_{S_C}]$ for WT, the expected-parts formula $T_{\mathrm{sim}}/(T+S+E)$ behind WTJ's controlled jump factor $f$, the greedy lowest-fill/highest-fill switch policy that matches the optimal distribution $\rho_i$, and the max-min integer program for worker allocation. These optima act as certificates: a learned policy is judged by how close its reward comes to a value that does not depend on the RL implementation.

What would settle it

Take the WT scenario and replace the exponential delay with a lognormal or Gamma distribution having the same mean and variance, then recompute Equation (3) and the simulated optimal reward; if the gap between learned policies and the paper's "optimal waiting time" changes by more than the noise in Table 1, the exponential assumption is decision-relevant. Additionally, on the publicly available production-line dataset used in Appendix D, a tail-sensitive distribution test at each station (rather than pooled 100-part averages) would show whether the exponential fits are artifacts of smoothing.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that active production-line control can be decomposed into a small set of measurable subproblems, and that with the LineFlow simulator these subproblems become standard RL environments with known optimal answers. It reports that on the static waiting-time (WT), jumping waiting-time (WTJ), worker-assignment (WA k,3k), and part-distribution (PD k) scenarios, the best learned policies approach the computed optima—recurrent PPO comes closest on WTJ, policy-gradient methods succeed at PD, and A2C is competitive on WA—while on the complex line (CL) every from-scratch agent collapses into a deadlock and even curriculum-trained recurrent PPO achieves lower reward than the paper's heuristic.

Load-bearing premise

The whole benchmark assumes each station's process time is a minimum plus an exponentially distributed random delay; if real stations follow other distributions or have setup-dependent times, the claimed optima are no longer the right targets.

Editorial extensions

If this is right

  • Standardized benchmarks: future RL work on active line control can say exactly how far from optimal a new algorithm is, because each LineFlow scenario ships with its own computable optimum, not just a heuristic baseline.
  • Memory matters: within these experiments, recurrent policies are the only ones that track a jumping bottleneck or manage the complex line, so non-Markovian observations are a first-class requirement for production control.
  • Curriculum is not optional on hard layouts: from-scratch agents in CL converge to a deadlock with zero reward, and annealing scrap penalties lets training escape that failure mode.
  • The remaining gap is stated openly: in the complex line the best learned agents score below the hand-tuned heuristic, which locates the open problem at reward shaping, hierarchical control, and multi-timescale memory rather than raw sample efficiency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if real stations deviate from the exponential-delay model (setup-dependent times, heavy tails, worker-dependent speeds), the closed-form optima in the paper are no longer the right yardsticks; a line operator should re-derive the waiting-time and distribution formulas for their own fitted distributions before trusting the benchmark numbers.
  • Editorial inference: the appendix's real-data validation pools 100 consecutive parts into averaged processing times and assumes negligible traversal times and round-robin routing, which weakens the directness of the sim-to-real evidence; a stricter test would compare full inter-completion distributions, not cumulative produced parts alone.
  • Editorial inference: a natural next experiment is to treat the heuristic that beats RL on CL as a teacher, using its decisions as demonstrations or as a shaping reward; that would test whether the remaining gap is a search problem or a representation problem.
  • Editorial inference: since the complex line combines exactly the subproblems that RL solves separately, another test is whether hierarchical decomposition—separate policies for waiting time, routing, and worker allocation coordinated by a meta-controller—reaches or exceeds the heuristic; the paper leaves this route open.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. LineFlow is an open-source Python discrete-event simulator for production lines, integrated with Gymnasium and Stable-Baselines3. The paper defines four analytically tractable control subproblems—optimal waiting time (WT), waiting time with jumps (WTJ), optimal part distribution (PD_k), and optimal worker assignment (WA_{k,N})—plus a combined complex line (CL). For each subproblem, the authors supply an "Optimal" column derived from modeling assumptions, and benchmark PPO, recurrent PPO, TRPO, and A2C against it. The experiments show that RL policies approach the computed optima on WT, PD, and WA, while only recurrent PPO approaches the WTJ comparator; on CL, RL agents converge to deadlock unless a curriculum is used, and a hand-designed heuristic outperforms them. An appendix reconstructs a Bosch production line from public data and reports a close match between simulated and real output.

Significance. The main value of this work is infrastructural: a standardized, extensible simulation environment for RL research on production-line control, with open-source code and clearly stated stochastic assumptions (T = T + ExpS). The worker-assignment optima are verified by exhaustive enumeration for k=3,4,5, and the PD optimum has a clean closed form; these are genuine strengths. However, the headline claim that learned policies "approach optimal performance in well-understood scenarios" is only as strong as the "Optimal" comparators, and the WTJ comparator is not an optimum. If the baseline issue is fixed by re-labeling or by a true upper bound, the benchmark would still be a useful contribution to the community.

major comments (3)
  1. [Section B.1, Table 1, Figure 13] The WTJ "Optimal" value 114.8±0.9 is not the outcome of an optimality proof. It is obtained by an agent that estimates E[T_A] from a rolling mean of the last l observed processing times, with l=1 selected by trying values and keeping the one with highest reward (Figure 13). This is an online heuristic tuned on the same benchmark, not an upper bound; a change-point detector or Bayesian online estimator could plausibly exceed 114.8±0.9. Because Table 1 labels this column "Optimal" and the abstract generalizes from it, the claim that RL approaches optimality in WTJ is unsupported. Please replace this baseline with a provable upper bound or relabel it (e.g., "rolling-mean baseline") and revise the abstract and Section 6 accordingly.
  2. [Section D, Figures 22-23] The real-data validation is weaker than presented: the exponential processing-time parameters are fitted to the Bosch dataset and the simulation is then compared against the very same dataset. This is a consistency check, not an independent validation of the exponential assumption or of the auxiliary assumptions (negligible traversal times, round-robin routing). The close parts-count match could reflect the fitted parameters rather than model validity. Please add a holdout evaluation (e.g., fit on the first portion of the production run and compare on the remainder), a sensitivity analysis, or an explicit caveat that this section is an illustration of reconstruction rather than evidence of sim-to-real transfer.
  3. [Section B.1, Eqs. (3)-(4)] The WT optimum is also stated rather than proved. The formula T*_W = E[T_A + 2T_g - T_SC] and the expected maximum parts in Eq. (4) are derived from a gap-filling argument under the assumption that A is the bottleneck, but no argument shows that no other (adaptive) policy can do better. The empirical reward curve in Figure 11 is supportive, but the section is titled "Optimality Proofs". Please supply the formal argument or soften the terminology to "analytic baseline".
minor comments (6)
  1. [Appendix A.2] The code example is missing a comma after the "Switch" dictionary entry; the `return` statement as printed would not parse.
  2. [Section 2.1] The symbol T is overloaded: it denotes both the processing-time random variable and the minimum processing time in "T = T + ExpS". Please introduce a distinct symbol such as T_min to improve readability of Eqs. (3)-(5).
  3. [Table 1] The parenthetical "maximal reward" values and the note that RL agents may outperform the expected optimal mean in individual episodes could confuse readers; recommend a separate column and a clearer definition of which statistic is reported and how it was computed over seeds and evaluation episodes.
  4. [Section 5.3] The sentence beginning "Our heuristics prioritized the buffer with the lowest fill level fills..." is grammatically incomplete and should be rewritten for clarity.
  5. [Section D] The description of the rolling, non-overlapping window of 100 consecutive parts should state explicitly how the histogram data in Figure 22 were aggregated and how the exponential parameters were estimated from the pooled averages.
  6. [Section 3.3] The claim that discrete-time interaction can approximate continuous-time control "with high fidelity" is not quantified; please report the Tstep values used in each benchmark (Tstep=1 appears in Table 7) and any evidence about discretization error.

Circularity Check

1 steps flagged · score 4.0 of 10

WTJ's 'Optimal' value is a tuned rolling-mean baseline rather than a proven optimum, so the 'approach optimal' claim for that scenario is partially self-referential; the other scenario optima are independently derived.

  1. fitted input called prediction [Section B.1 (Waiting Time, WTJ paragraph), reported in Table 1, WTJ row]
    "To get an optimal policy for WTJ, we estimate E[TA + 2·Tg−TSC] from Equation (3) by regressing on E[TA] with the processing times reported from A. Essentially, we take a rolling mean of the last l processing times observations from A. ... By varying l and testing the agent for WTJ (see Figure 13), we found that l = 1 gives the best reward. This reward is used as optimal value in Section 5."

    The WTJ 'Optimal' yardstick in Table 1 (114.8±0.9) is not a derived upper bound but the reward of a rolling-mean heuristic whose look-back l was selected by maximizing reward on the very WTJ benchmark it is then used to judge. Thus the claim that RL 'approaches optimal performance' in WTJ compares learned policies to a self-tuned baseline, not to an independent optimum. A different adaptive estimator could plausibly exceed this value, so the benchmark target is defined by the baseline's best tuned performance rather than by an analytic optimum.

full rationale

The core benchmark is largely non-circular. The WT optimum follows from the explicit balance condition T*_W = E[TA + 2·Tg − TSC] (Eq. 3) and the expected throughput expression (Eq. 4); the PD optimum follows from the throughput formula E[N] = sum_i Tsim/((1+Si)Ti) (Eq. 5); and the WA optimum follows from the min-max integer program (Eq. 7). These analytic optima are verified against enumerated simulations (e.g., Figure 15) and are independent of the RL agents' fitted parameters. The real-data validation in Section D fits exponential distributions to the Bosch dataset and then compares the same dataset's output to simulation; this is in-sample calibration rather than derivation, so it weakens external validity but is not circular in the load-bearing sense. There is no self-citation chain or imported uniqueness theorem. The only substantive circularity is the WTJ baseline: its 'optimal' value is selected by tuning l on the benchmark reward itself, making the WTJ 'approaches optimal' conclusion partially self-referential. Since WTJ is one of four headline scenarios, this raises the score to 4 rather than higher.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces a software artifact (LineFlow) and a reward function Cπ(t), but no new physical entity, conservation law, or unobserved mediator. The hand-chosen parameters above are the main external inputs.

free parameters (4)
  • Performance coefficient c in worker assignment = 0.3
    Section 4.3 models worker effect as pc(n)=exp(-c n); c=0.3 is hand-chosen and directly determines the optimal partition in WA. The text also misstates exp(-0.3) as a 74% reduction instead of a reduction to 74%.
  • WTJ online-estimator look-back l = 1
    Section B.1 tunes l over {1,...,9} and selects l=1 because it maximizes benchmark reward; this tuned estimator is then used as the 'optimal' WTJ baseline.
  • WTJ reward-fraction R = Only specified as 0.5<R<1.0
    R sets the target maximum reward for WTJ episodes; the paper never states the value used in experiments and the f formula can become negative unless R is large enough relative to Tjump/Tsim.
  • Rolling pooling window for real-data processing times = 100 parts
    Section D averages processing times over windows of 100 consecutive parts to compensate for timestamp resolution; this pooling choice shapes the fitted exponential distributions.
assumptions (5)
  • domain assumption Processing times at each station are distributed as T + ExpS, with a fixed minimum plus an exponential.
    Section 2.1 states this for all stations, and every optimality formula in Section 4 and the Section D validation build on it.
  • domain assumption In PD, source and sink processing times are negligible and get/put times are already included in T_i.
    Section 4.2 uses this to derive E[N_i]=Tsim/((1+S_i)T_i) and the routing fractions rho_i.
  • domain assumption In WT/WTJ, assembly A is the bottleneck, so total throughput is governed by A's cycle.
    Section 4.1 states this before deriving T*_W; if A were not the bottleneck, Equation (3) would not give an optimal waiting time.
  • ad hoc to paper Worker effect follows the exponential law pc(n)=exp(-c n) with c=0.3.
    Section 4.3 introduces this law for the WA case study; it is not derived from worker data and c is hand-picked.
  • ad hoc to paper For the Bosch-data layout, traversal times are negligible and switches route round-robin.
    Section D makes these assumptions because the dataset lacks buffer and traversal information, then compares the resulting simulation against the same dataset.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LineFlow: A Framework to Learn Active Control of Production Lines." pith.science (2026). https://pith.science/paper/27UDU4UO

@misc{pith2026250506744,
  author       = {Pith},
  title        = {Pith review of: LineFlow: A Framework to Learn Active Control of Production Lines},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/27UDU4UO}},
  note         = {Machine review of arXiv:2505.06744}
}
read the original abstract

Many production lines require active control mechanisms, such as adaptive routing, worker reallocation, and rescheduling, to maintain optimal performance. However, designing these control systems is challenging for various reasons, and while reinforcement learning (RL) has shown promise in addressing these challenges, a standardized and general framework is still lacking. In this work, we introduce LineFlow, an extensible, open-source Python framework for simulating production lines of arbitrary complexity and training RL agents to control them. To demonstrate the capabilities and to validate the underlying theoretical assumptions of LineFlow, we formulate core subproblems of active line control in ways that facilitate mathematical analysis. For each problem, we provide optimal solutions for comparison. We benchmark state-of-the-art RL algorithms and show that the learned policies approach optimal performance in well-understood scenarios. However, for more complex, industrial-scale production lines, RL still faces significant challenges, highlighting the need for further research in areas such as reward shaping, curriculum learning, and hierarchical control.

Figures

Figures reproduced from arXiv: 2505.06744 by the authors.

Figure 1
Figure 1. Active line control based on real-time data. active line control systems typically combine rule-based strategies and mathematical models to optimize schedul￾ing and resource allocation, focusing mostly on bottleneck identification, see for instance (Roser et al., 2014; 2017; Li et al., 2009; Lai et al., 2021) and references therein. Find￾ing optimal control strategies is a complex task that not only requires a detai… view at source ↗
Figure 2
Figure 2. A production line visualized with LineFlow. times of the stations and their interplay. To be more precise, the processing times are assumed to be exponentially dis￾tributed like in (Bierbooms, 2012) as T = T +ExpS, where T ≥ 0 is the minimal processing time possible and where ExpS denotes the exponential distribution2 with mean S. We distinguish different types of stations, among them are sources that set up compone… view at source ↗
Figure 3
Figure 3. Representatives of the three atomic production line chal￾lenges analyzed in this case study. 3.3. Implementation Details LineFlow is designed to address key challenges in opti￾mizing active line controls while meeting the needs of RL researchers. The discrete event simulation, handling station interactions and stochasticity, is built on SimPy (SimPy, 2025). Its object-oriented structure enables easy customiza￾tion o… view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: The jumps in the processing time of the assembly A in WTJ for different simulations of length 4000. expiration time TAC called assembly condition: If the time from their setup at SC to the time the process starts at A is larger than TAC, then A has to dispose this comp…
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Evaluation reward over global steps for PD3. PPO TRPO rPPO A2C Optimal WT 155.0 ± 3.0 (158.4) 157.1 ± 1.9 (158.6) 157.7 ± 1.3 (159.0) 154.4 ± 5.9 (158.6) 156.2 ± 1.5 WTJ 93.7 ± 6.9 (101.4) 93.9 ± 7.6 (102.4) 105.9 ± 11.6 (113.0) 100.5 ± 11.1 (110.2) 114.8 ± 0.9 WA3,9 2…
Figure 8
Figure 8. Figure 8: Upper figure shows a comparison of recurrent PPO trained with (blue) and without (red) curriculum using three differ￾ent seeds. Lower figure shows a comparison of the stacked PPO trained using curriculum with a stack of 40 (red) and 100 (blue). parameters have been use…
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: Selection of features extracted from the line displayed in [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Some layouts implemented in LineFlow. B. Optimality Proofs for Case Studies In this section, we give proofs for optimal control policies for the scenarios WT and WTJ in Section B.1, PD in Section B.2, and WA in Section B.3. In Section B.4, we state a heuristic for CL …
Figure 11
Figure 11. Figure 11: Visualisation of the setting in WT with optimal waiting time T ∗ W = 18.5 as in Equation (3) and the maximal number of expected parts as computed in Equation (4). Next, we explain the construction of f and Tjump of WTJ with a jumping processing time of the assembly A.…
Figure 12
Figure 12. Figure 12: Number of parts produced for WTJ and WT with large assembly condition TAC and no waiting at SC for varying R. To get an optimal policy for WTJ, we estimate E[TA + 2 · Tg − TSC ] from Equation (3) by regressing on E[TA] with the processing times reported from A. Essent…
Figure 13
Figure 13. Figure 13: ), we found that l = 1 gives the best reward. This reward is used as optimal value in Section 5. 1 2 3 4 5 6 7 8 9 Look-back l 100 105 110 115 120 Reward [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 15
Figure 15. Figure 15: Performance of all possible monotone partitions for Ti = (16 + i · 4), and c = 0.3 for a simulation length of 2000 in the worker distribution example. 0.0 0.1 0.2 0.3 0.4 0.5 P1 P2 P3 Part distributions greedy optimum 0.0 0.1 0.2 0.3 0.4 P1 P2 P3 P4 Part distributions…
Figure 14
Figure 14. Figure 14: Part distributions of the greedy policy compared with optimal distribution for k ∈ {3, 4, 5}. B.3. Worker Assignments In this section, we formulate T ∗ C = min (n1,...,nk)∈NN,k max i∈[k] E[TTi,Si,ni ] as given in Equation (2) as integer optimization problem. Recall th…
Figure 16
Figure 16. Figure 16: Best performance of algorithms on evaluation environments for WT and WTJ. 50000 100000 150000 200000 250000 300000 350000 400000 Global step 50 0 50 100 150 Reward A2C PPO RecurrentPPO TRPO (a) WT 0.2 0.4 0.6 0.8 1.0 Global step 1e7 75 50 25 0 25 50 75 100 Reward A2C …
Figure 17
Figure 17. Figure 17: Reward over steps for WT and WTJ. C.2. Part Distribution 100 200 300 400 500 600 Best rewards A2C (lr=0.0005, ent_c=0.0) A2C (lr=0.0005, ent_c=0.001) A2C (lr=0.001, ent_c=0.0) A2C (lr=0.001, ent_c=0.001) A2C (lr=0.01, ent_c=0.0) A2C (lr=0.01, ent_c=0.001) PPO (lr=0.00…
Figure 18
Figure 18. Figure 18: Best performance of algorithms on evaluation environments for PDk. 0.2 0.4 0.6 0.8 1.0 Global step 1e6 200 300 400 500 Reward A2C PPO RecurrentPPO TRPO (a) k = 3 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Global step 1e6 200 300 400 500 600 Reward A2C PPO RecurrentPPO TRPO (b) k…
Figure 19
Figure 19. Figure 19: Reward over steps for PDk. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: Best performance of algorithms on evaluation environments for WAk,3k. 50000 100000 150000 200000 250000 300000 350000 400000 Global step 100 125 150 175 200 225 250 275 Reward A2C PPO RecurrentPPO TRPO (a) k = 3 100000 200000 300000 400000 500000 Global step 100 120 1…
Figure 21
Figure 21. Figure 21: Reward over steps for WAk,3k. D. Validation on Real Production Data In this section, we evaluate the sim-to-real gap of LineFlow using the publicly available production line dataset from (Risdal et al., 2016). In this dataset, each row corresponds to a produced part, …
Figure 22
Figure 22. Figure 22: Histograms of the averaged processing times in a log scale together with a fitted exponential distribution in red. Next, we implemented the reconstructed layout in LineFlow and with the aim to compare the number of parts produced in the simulation with those observed …
Figure 23
Figure 23. Figure 23: The implemented layout in LineFlow (left) and its simulated output compared with the output of the real production line (right). 24 [PITH_FULL_IMAGE:figures/full_fig_p024_23.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 32 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Ali, A. M. and Tirel, L. Action masked deep reinforcement learning for controlling industrial assembly lines. In 2023 IEEE World AI IoT Congress (AIIoT), pp.\ 0797--0803, 2023. doi:10.1109/AIIoT58121.2023.10174426

    arXiv 2023

  3. [3]

    Gekko optimization suite

    Beal, L., Hill, D., Martin, R., and Hedengren, J. Gekko optimization suite. Processes, 6 0 (8): 0 106, 2018. doi:10.3390/pr6080106

    doi:10.3390/pr6080106 2018

  4. [4]

    Performance analysis of production lines: Discrete and continuous flow models

    Bierbooms, R. Performance analysis of production lines: Discrete and continuous flow models. PhD Thesis, Technische Universiteit Eindhoven, 2012

    work page 2012

  5. [5]

    Modeling and control of dispensing processes for surface mount technology

    Chen, X., Schoenau, G., and Zhang, W. Modeling and control of dispensing processes for surface mount technology. IEEE/ASME Transactions on Mechatronics, 10: 0 326--334, 2005. doi:10.1109/TMECH.2005.848295

    work page arXiv 2005

  6. [6]

    Y., Malyutin, S., and Soukhal, A

    Dolgui, A., Kovalev, S., Kovalyov, M. Y., Malyutin, S., and Soukhal, A. Optimal workforce assignment to operations of a paced assembly line. European Journal of Operational Research, 264 0 (1): 0 200--211, 2018. doi:10.1016/j.ejor.2017.06.017. URL https://www.sciencedirect.com/science/article/pii/S0377221717305350

    work page doi:10.1016/j.ejor.2017.06.017 2018

  7. [7]

    Deep reinforcement learning for optimal planning of assembly line maintenance

    Geurtsen, M., Adan, I., and Atan, Z. Deep reinforcement learning for optimal planning of assembly line maintenance. Journal of Manufacturing Systems, 69: 0 170--188, 2023. doi:https://doi.org/10.1016/j.jmsy.2023.05.011. URL https://www.sciencedirect.com/science/article/pii/S0278612523000845

    work page doi:10.1016/j.jmsy.2023.05.011 2023

  8. [8]

    N., Cortez, P., Carvalho, M

    Gonçalves, J. N., Cortez, P., Carvalho, M. S., and Frazão, N. M. A multivariate approach for multi-step demand forecasting in assembly industries: Empirical evidence from an automotive supply chain. Decision Support Systems, 142: 0 113452, 2021. doi:10.1016/j.dss.2020.113452. URL https://www.sciencedirect.com/science/article/pii/S0167923620302074

    arXiv 2021

Show all 57 references
  1. [9]

    A reinforcement learning decision model for online process parameters optimization from offline data in injection molding

    Guo, F., Zhou, X., Liu, J., Zhang, Y., Li, D., and Zhou, H. A reinforcement learning decision model for online process parameters optimization from offline data in injection molding. Applied Soft Computing, 85: 0 105828, 2019. doi:10.1016/j.asoc.2019.105828. URL https://www.sc...

    2019

  2. [10]

    R., Millman, K

    Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., Haldane, A., del R \' i o, J. F., Wiebe, M., Peterson, P., G \' e rard...

    2020 doi

  3. [11]

    J., Lillicrap, T

    Heess, N., Hunt, J. J., Lillicrap, T. P., and Silver, D. Memory-based control with recurrent neural networks, 2015. URL https://arxiv.org/abs/1512.04455

    2015 arXiv

  4. [12]

    Y., and Jiang, J

    Huang, B., Wang, W., Ren, S., Zhong, R. Y., and Jiang, J. A proactive task dispatching method based on future bottleneck prediction for the smart factory. International Journal of Computer Integrated Manufacturing, 32 0 (3): 0 278--293, 2019

    2019

  5. [13]

    An intelligent weld control strategy based on reinforcement learning approach

    Jin, Z., Li, H., and Gao, H. An intelligent weld control strategy based on reinforcement learning approach. The International Journal of Advanced Manufacturing Technology, 100 0 (9): 0 2163--2175, 2019. doi:10.1007/s00170-018-2864-2

    2019 doi

  6. [14]

    M., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Ž \'i dek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S

    Jumper, J. M., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Ž \'i dek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Pete...

    2021

  7. [15]

    Machine learning applications in production lines: A systematic literature review

    Kang, Z., Catal, C., and Tekinerdogan, B. Machine learning applications in production lines: A systematic literature review. Computers & Industrial Engineering, 149: 0 106773, 2020. doi:10.1016/j.cie.2020.106773. URL https://www.sciencedirect.com/science/article/pii/S036083522030485X

    2020

  8. [16]

    Kim, G. H. and Lee, C. S. G. Genetic reinforcement learning approach to the heterogeneous machine scheduling problem. IEEE Transactions on Robotics and Automation, 14 0 (6): 0 879--893, 1998. doi:10.1109/70.736772

    1998 doi

  9. [17]

    Designing an adaptive production control system using reinforcement learning

    Kuhnle, A., Kaiser, J.-P., Theiss, F., Stricker, N., and Lanza, G. Designing an adaptive production control system using reinforcement learning. Journal of Intelligent Manufacturing, 32 0 (3): 0 855--876, 2021. doi:10.1007/s10845-020-01612-y

    2021 doi

  10. [18]

    Data-driven dynamic bottleneck detection in complex manufacturing systems

    Lai, X., Shui, H., Ding, D., and Ni, J. Data-driven dynamic bottleneck detection in complex manufacturing systems. Journal of Manufacturing Systems, 60: 0 662--675, 2021. doi:10.1016/j.jmsy.2021.07.016. URL https://www.sciencedirect.com/science/article/pii/S0278612521001539

    2021 doi

  11. [19]

    Real time production improvement through bottleneck control

    Li, L., Chang, Q., Ni, J., and Biller, S. Real time production improvement through bottleneck control. International Journal of production research, 47 0 (21): 0 6145--6158, 2009

    2009

  12. [20]

    T., Tan, B

    Liberopoulos, G., Papadopoulos, C. T., Tan, B. s., Smith, J. M., and Gershwin, S. B. (eds.). Stochastic Modeling of Manufacturing Systems : Advances in Design , Performance Evaluation , and Control Issues . Springer, 2006. ISBN 978-3-540-26579-5 978-3-540-29057-5. doi:10.1007/...

    2006 doi

  13. [21]

    C., Schäfer, L., Matta, A., and Lanza, G

    Loffredo, A., May, M. C., Schäfer, L., Matta, A., and Lanza, G. Reinforcement learning for energy-efficient control of parallel and identical machines. CIRP Journal of Manufacturing Science and Technology, 44: 0 91--103, 2023. doi:10.1016/j.cirpj.2023.05.007. URL https://www.s...

    2023 doi

  14. [22]

    C., Matta, A., and Lanza, G

    Loffredo, A., May, M. C., Matta, A., and Lanza, G. Reinforcement learning for sustainability enhancement of production lines. Journal of Intelligent Manufacturing, 35 0 (8): 0 3775--3791, dec 2024. doi:10.1007/s10845-023-02258-2

    2024 doi

  15. [23]

    The impact of industry 4.0 on bottleneck analysis in production and manufacturing: Current trends and future perspectives

    Mahmoodi, E., Fathi, M., and Ghobakhloo, M. The impact of industry 4.0 on bottleneck analysis in production and manufacturing: Current trends and future perspectives. Computers & Industrial Engineering, 174: 0 108801, 2022. doi:10.1016/j.cie.2022.108801. URL https://www.scienc...

    2022

  16. [24]

    Masinelli, G., Le-Quang, T., Zanoli, S., Wasmer, K., and Shevchik, S. A. Adaptive laser welding control: A reinforcement learning approach. IEEE Access, 8: 0 103803--103814, 2020. doi:10.1109/ACCESS.2020.2998052

    2020

  17. [25]

    P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K

    Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. Asynchronous methods for deep reinforcement learning. In Balcan, M. F. and Weinberger, K. Q. (eds.), Proceedings of The 33rd International Conference on Machine Learning, ...

    1928

  18. [26]

    Introduction to TPM: Total Productive Maintenance

    Nakajima, S. Introduction to TPM: Total Productive Maintenance. Preventative Maintenance Series. Productivity Press, 1988. ISBN 9780915299232. URL https://books.google.de/books?id=XKc28H3JeUUC

    1988

  19. [27]

    E., and Stone, P

    Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M. E., and Stone, P. Curriculum learning for reinforcement learning domains: A framework and survey. Journal of Machine Learning Research, 21 0 (181): 0 1--50, 2020. URL http://jmlr.org/papers/v21/20-212.html

    2020

  20. [28]

    A review on reinforcement learning: Introduction and applications in industrial process control

    Nian, R., Liu, J., and Huang, B. A review on reinforcement learning: Introduction and applications in industrial process control. Computers & Chemical Engineering, 139: 0 106886, 2020. doi:10.1016/j.compchemeng.2020.106886. URL https://www.sciencedirect.com/science/article/pii...

    2020

  21. [29]

    C., Kuhnle, A., and Lanza, G

    Overbeck, L., Hugues, A., May, M. C., Kuhnle, A., and Lanza, G. Reinforcement learning based production control of semi-automated manufacturing systems. Procedia CIRP, 103: 0 170--175, 2021. doi:10.1016/j.procir.2021.10.027. URL https://www.sciencedirect.com/science/article/pi...

    2021 doi

  22. [30]

    Irizarry, M., Resto, P., and Mej \' a, H

    Padr \'o n, M., de los A. Irizarry, M., Resto, P., and Mej \' a, H. P. A methodology for cost-oriented assembly line balancing problems. Journal of Manufacturing Technology Management, 20 0 (8): 0 1147--1165, 2009

    2009

  23. [31]

    Generalization, mayhems and limits in recurrent proximal policy optimization, 2022

    Pleines, M., Pallasch, M., Zimmer, F., and Preuss, M. Generalization, mayhems and limits in recurrent proximal policy optimization, 2022. URL https://arxiv.org/abs/2205.11104

    2022 arXiv

  24. [32]

    Memory gym: Partially observable challenges to memory-based agents

    Pleines, M., Pallasch, M., Zimmer, F., and Preuss, M. Memory gym: Partially observable challenges to memory-based agents. In International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=jHc8dCx6DDr

    2023

  25. [33]

    M., Ou, W., Yenradee, P., and Huynh, V.-N

    Pérez-Wheelock, R. M., Ou, W., Yenradee, P., and Huynh, V.-N. A demand-driven model for reallocating workers in assembly lines. IEEE Access, 10: 0 80300--80320, 2022. doi:10.1109/ACCESS.2022.3194658

    2022

  26. [34]

    Stable-baselines3: Reliable reinforcement learning implementations

    Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22 0 (268): 0 1--8, 2021. URL http://jmlr.org/papers/v22/20-1364.html

    2021

  27. [35]

    Bosch production line performance

    Risdal, M., Prasanth, RumiGhosh, soundar, W., S., and Cukierski, W. Bosch production line performance. https://kaggle.com/competitions/bosch-production-line-performance, 2016. Kaggle

    2016

  28. [36]

    Shifting bottleneck detection

    Roser, C., Nakano, M., and Tanaka, M. Shifting bottleneck detection. In Proceedings of the Winter Simulation Conference. IEEE, 2003

    2003

  29. [37]

    Reliable shop floor bottleneck detection for flow lines through process and inventory observations

    Roser, C., Lorentzen, K., and Deuse, J. Reliable shop floor bottleneck detection for flow lines through process and inventory observations. Procedia CIRP, 19: 0 63--68, 2014. doi:10.1016/j.procir.2014.05.020. URL https://www.sciencedirect.com/science/article/pii/S2212827114006...

    2014 doi

  30. [38]

    Bottleneck prediction using the active period method in combination with buffer inventories

    Roser, C., Lorentzen, K., Lenze, D., Deuse, J., Klenner, F., Richter, R., Schmitt, J., and Willats, P. Bottleneck prediction using the active period method in combination with buffer inventories. In L \"o dding, H., Riedel, R., Thoben, K.-D., von Cieminski, G., and Kiritsis, D...

    2017

  31. [39]

    and Becker, C

    Scholl, A. and Becker, C. State-of-the-art exact and heuristic solution procedures for simple assembly line balancing. European Journal of Operational Research, 168 0 (3): 0 666--693, 2006

    2006

  32. [40]

    Trust region policy optimization

    Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. Trust region policy optimization. In Bach, F. and Blei, D. (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp.\ 1889--1897, Lil...

    2015

  33. [41]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  34. [42]

    skrl: Modular and flexible library for reinforcement learning

    Serrano-Muñoz, A., Chrysostomou, D., Bøgh, S., and Arana-Arexolaleiba, N. skrl: Modular and flexible library for reinforcement learning. Journal of Machine Learning Research, 24 0 (254): 0 1--9, 2023. URL http://jmlr.org/papers/v24/23-0112.html

    2023

  35. [43]

    Intelligent scheduling of discrete automated production line via deep reinforcement learning

    Shi, D., Fan, W., Xiao, Y., Lin, T., and Xing, C. Intelligent scheduling of discrete automated production line via deep reinforcement learning. International Journal of Production Research, 58 0 (11): 0 3362--3380, 2020. doi:10.1080/00207543.2020.1717008

    2020

  36. [44]

    Real-time scheduling for a smart factory using a reinforcement learning approach

    Shiue, Y.-R., Lee, K.-C., and Su, C.-T. Real-time scheduling for a smart factory using a reinforcement learning approach. Computers & Industrial Engineering, 125: 0 604--614, 2018. doi:10.1016/j.cie.2018.03.039. URL https://www.sciencedirect.com/science/article/pii/S036083521830130X

    2018 doi

  37. [45]

    J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al

    Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. Mastering the game of go with deep neural networks and tree search. Nature, 529 0 (7587): 0 484--489, 2016

    2016

  38. [46]

    Simpy 4.1 webpage, 2025

    SimPy, T. Simpy 4.1 webpage, 2025. URL https://simpy.readthedocs.io/en/latest/

    2025

  39. [47]

    A., Th \"u rer, M., and Chang, Q

    Subramaniyan, M., Skoogh, A., Bokrantz, J., Sheikh, M. A., Th \"u rer, M., and Chang, Q. Artificial intelligence for throughput bottleneck analysis--state-of-the-art and future directions. Journal of Manufacturing Systems, 60: 0 734--751, 2021

    2021

  40. [48]

    Sutton, R. S. and Barto, A. Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning. The MIT Press, second edition edition, 2020. ISBN 978-0-262-03924-6

    2020

  41. [49]

    and Chauhan, S

    Szarski, M. and Chauhan, S. Instant flow distribution network optimization in liquid composite molding using deep reinforcement learning. Journal of Intelligent Manufacturing, 34 0 (1): 0 197--218, 2023. doi:10.1007/s10845-022-01990-5

    2023 doi

  42. [50]

    A parallel deep reinforcement learning framework for controlling industrial assembly lines

    Tortorelli, A., Imran, M., Delli Priscoli, F., and Liberati, F. A parallel deep reinforcement learning framework for controlling industrial assembly lines. Electronics, 11 0 (4), 2022. doi:10.3390/electronics11040539. URL https://www.mdpi.com/2079-9292/11/4/539

    2022 doi

  43. [51]

    U., Cola, G

    Towers, M., Kwiatkowski, A., Terry, J., Balis, J. U., Cola, G. D., Deleu, T., Goulão, M., Kallinteris, A., Krimmel, M., KG, A., Perez-Vicente, R., Pierré, A., Schulhoff, S., Tai, J. J., Tan, H., and Younis, O. G. Gymnasium: A standard interface for reinforcement learning envir...

    2024 arXiv

  44. [52]

    Viharos, Z. J. and Jakab, R. Reinforcement learning for statistical process control in manufacturing. Measurement, 182, 2021. doi:10.1016/j.measurement.2021.109616. URL https://www.sciencedirect.com/science/article/pii/S0263224121005881

    2021

  45. [53]

    M., Mathieu, M., Dudzik, A., Chung, J., Choi, D

    Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., Oh, J., Horgan, D., Kroiss, M., Danihelka, I., Huang, A., Sifre, L., Cai, T., Agapiou, J. P., Jaderberg, M., Vezhnevets, A. S., Leblond, R., Po...

    2019

  46. [54]

    Multi-agent reinforcement learning based maintenance policy for a resource constrained flow line system

    Wang, X., Wang, H., and Qi, C. Multi-agent reinforcement learning based maintenance policy for a resource constrained flow line system. Journal of Intelligent Manufacturing, 27 0 (2): 0 325--333, 2016. doi:10.1007/s10845-013-0864-5

    2016 doi

  47. [55]

    D ata S tructures for S tatistical C omputing in P ython

    W es M c K inney. D ata S tructures for S tatistical C omputing in P ython. In S t\'efan van der W alt and J arrod M illman (eds.), P roceedings of the 9th P ython in S cience C onference , pp.\ 56 -- 61, 2010. doi:10.25080/Majora-92bf1922-00a

    2010 doi

  48. [56]

    Daydreamer: World models for physical robot learning

    Wu, P., Escontrela, A., Hafner, D., Abbeel, P., and Goldberg, K. Daydreamer: World models for physical robot learning. In Liu, K., Kulic, D., and Ichnowski, J. (eds.), Proceedings of The 6th Conference on Robot Learning, volume 205 of Proceedings of Machine Learning Research, ...

    2023

  49. [57]

    and Lan, R

    Zou, Y. and Lan, R. An end-to-end calibration method for welding robot laser vision systems with deep reinforcement learning. IEEE Transactions on Instrumentation and Measurement, 69 0 (7): 0 4270--4280, 2020. doi:10.1109/TIM.2019.2942533

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.