{"id":"67ef0d5b-510d-4414-a424-57dbe03f1e5a","arxiv_id":"1909.02180","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"LLP-GAN trains a GAN discriminator to be the instance classifier using only bag-level label proportions, and its optimal discriminator is shown to be a bag-proportion mixture.","lead":"This paper proposes LLP-GAN, a generative adversarial network that learns an image classifier from only bag-level class proportions, such as knowing that a group contains 45% dolphins and 30% pandas. The authors prove properties of the adversarial objective and report accuracy gains over earlier label-proportion methods on several benchmarks, though the strongest concurrent baseline is not compared directly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The optimality theorems characterize the Jensen lower-bound surrogate (Eq. 6), not the actual bag-proportion loss, and no gap bound is supplied; this is the load-bearing weakness in the global-optimality claim.","rationale":"The reader's weakest assumption is on target. I looked for an even more basic algebraic flaw in Theorem 2; the variational calculation in Eqs. (10)-(13) is internally consistent for ~V, with the usual caveat that p_g may be degenerate, so the main gap is external validity. The feature-matching change in Remark 6 means the generator algorithm does not literally minimize C(G), but this weakens only the generator-side equilibrium, not the normalized posterior (Eq. 11), which is independent of p_g; hence it is not the single most load-bearing issue. The surrogate gap, by contrast, directly affects the classifier returned by the method: optimizing ~V can produce a different discriminator from optimizing V, and no argument shows the two coincide outside disjoint bag supports. Definition 1's disjoint partition is a real assumption and is violated by the random-bag experiments. The proposed two-bag Gaussian test settles whether the gap is material: if the Eq. (9) classifier already minimizes Lprop, the concern is refuted; if not, the theorem should be restated as optimality for the surrogate only. This is a condition for the paper's central theoretical claim, not a rejection of the empirical contribution. Code is provided and the empirical comparison is reasonable, which supports a conditional verdict rather than rejection.","tokens_in":12489,"tokens_out":17879,"duration_ms":197976,"concrete_test":"Construct a two-class synthetic LLP problem with two overlapping bags: bag 1 draws x ~ N(-1,1) with true proportion p1=(0.9,0.1); bag 2 draws x ~ N(1,1) with p2=(0.1,0.9). Compute the surrogate-optimal discriminator from Eq. (9) and evaluate the true bag-level loss Lprop = -sum_i sum_k p_i(k) log( mean_{x in bag i} pD(k|x) ). Then directly minimize Lprop over a flexible family, such as threshold or logistic functions of x, and evaluate the same loss. If the directly optimized classifier attains strictly lower Lprop than the Eq. (9) classifier, the lower-bound optimum is suboptimal for the true LLP objective, confirming that the theorems do not cover the objective of interest. A secondary check is to verify that the Jensen gap between the true Lprop and the lower-bound value at the Eq. (9) classifier is nonzero in this overlapping setting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the global optimality of C(G) = max_D ~V (Theorem 2) and the closed-form optimal discriminator (Theorem 1, Eq. 9). But ~V is not the LLP objective V of Eq. (4): Remark 2 replaces the bag-level cross-entropy -CEL(p,p) with its Jensen lower bound, swapping the log of an average for an average of logs. Theorems 1 and 2 therefore characterize the equilibrium of the surrogate, and Algorithm 1 optimizes this surrogate for D. No bound on the Jensen gap is provided. When bag supports overlap, which is the actual experimental protocol since bags are random subsets of the same training distribution, the surrogate-optimal normalized posterior is q_k(x) = sum_i p_i(k) p_i^d(x) / sum_i p_i^d(x), a density-weighted average of bag proportions rather than the per-bag proportion p_i(k). In that setting the inequality in Eq. (6) is strict, and the surrogate optimum need not maximize the true bag-level objective V. Definition 1 assumes a disjoint partition of the data space into bag sections, an assumption not satisfied by the random bags used in Section 4. Thus the proven global optimality does not transfer to the LLP loss that motivates the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LLP-GAN, a GAN-based method for learning from label proportions (LLP). The discriminator is trained with an adversarial real/fake loss plus a bag-level proportion cross-entropy term, which the authors replace by a Jensen lower bound to make the objective amenable to SGD. The generator is trained with feature matching. The main theoretical claims are an explicit optimal discriminator (Theorem 1, Eq. 9) and a global optimality result for the generator (Theorem 2) under a non-parametric, disjoint-bag idealization. Experiments on MNIST, SVHN, CIFAR-10, and CIFAR-100 report substantially lower test error than DLLP, especially for large bag sizes, together with comparisons to InvCal and alter-SVM in a binary subset.","tokens_in":12772,"tokens_out":16831,"duration_ms":175970,"significance":"If the theoretical claims were correct, the paper would be a useful contribution: it gives an end-to-end deep LLP algorithm with a direct instance-level classifier obtained from the discriminator, avoids parametric distributional assumptions, and shows consistent empirical gains over DLLP at large bag sizes. The authors also ship code and report results across four datasets and four bag sizes, including random-bag stability checks. However, the central theoretical results are not sound: the optimality theorems concern an unquantified surrogate objective, and the closed-form discriminator in Theorem 1 is not even a stationary point of that surrogate for the value of lambda used in the analysis. The empirical results may still indicate a useful heuristic, but the paper's stated theoretical foundation does not support its main claims.","major_comments":[{"comment":"Remark 2 replaces the actual bag-proportion cross-entropy -CEL(p, p_bar) by its Jensen lower bound, and all subsequent theorems are stated for the surrogate objective ~V, not for the LLP objective V in Eq. (4). Since Algorithm 1 trains the discriminator on ~V, and no bound on the Jensen gap is provided, the global optimality proven in Theorems 1 and 2 does not transfer to the loss that motivates the method. The paper needs either a quantitative gap bound or an explicit argument that the surrogate and the true objective share the same optima; neither is supplied.","section":"§3.1, Remark 2 and Eq. (6)"},{"comment":"The proposed optimal discriminator is not a maximizer of ~V. For fixed G, writing S = sum_k P_D(k|x), A_k = sum_i p_i(k) p_i^d(x), and p_d = sum_i p_i^d(x), the integrand in Eq. (10) is F = p_d log S + sum_k A_k log P_k + (p_g - p_d) log(1-S). At the claimed values P_k = A_k/(p_d+p_g) and S = p_d/(p_d+p_g), the derivative with respect to P_k evaluates to (p_d+p_g)^2 / p_g, not zero. The correct stationary condition gives S = 2p_d/(p_d+p_g) when lambda = 1, which is not a valid probability whenever p_d > p_g; in that regime ~V is unbounded above because (p_g - p_d) log(1-S) tends to +infinity as S tends to 1. Thus Eq. (9) and the results built on it are not valid as stated.","section":"§3.2, Theorem 1 and Eq. (9)"},{"comment":"The theoretical analysis assumes a fixed partition of the data space into disjoint sections, with each bag distribution p_i^d supported on its own section. The experiments instead form bags by randomly partitioning the finite training set, so the bag distributions all have the same (global) support. Under the surrogate objective, when the bag supports coincide, Eq. (11) gives a normalized posterior approximately equal to (1/n) sum_i p_i(k), i.e., the global class proportions independent of x. This contradicts the instance-level accuracy reported in Table 1 and shows that the experimental regime is not the regime covered by the optimality theorems.","section":"§3.2, Definition 1 and Lemma 1 vs. §4.1"},{"comment":"The derivation of C(G) is also internally inconsistent. In Eq. (13) the JSD term is written as JSD(p_d || p_g), but p_d = sum_i p_i^d is unnormalized (integral n) while p_g is normalized, so the standard Jensen-Shannon divergence is not defined for this pair. Evaluating Eq. (13) at the claimed optimum p_g = p_d/n gives an additional (n-1) log 2 term relative to the constant n log n - (n+1) log(n+1) stated in Eq. (14). The generator optimality calculation needs to be redone with normalized densities.","section":"§3.3, Eq. (13) and Eq. (14)"}],"minor_comments":[{"comment":"The inequality in Eq. (6) is written as '>' but Jensen's inequality gives '>='; the strict sign is not justified in general.","section":"§3.1, Eq. (6)"},{"comment":"The sentence 'Following a standard setting in the previous work [? 28]' contains an unresolved placeholder that must be fixed.","section":"Appendix §6"},{"comment":"The naming of the SVM baselines is inconsistent: 'alter-SVM' appears in the abstract and Figure 5, 'alter-pSVM' in Table 4, and 'alter-proportional-SVM' in the text; the notation should be unified.","section":"§4 and Appendix §7.1"},{"comment":"The table caption and column headers for the random bag-generation experiment are unclear: the number of random repetitions is mixed with the number of errors and the reported accuracy, which makes the stability claim hard to evaluate.","section":"Appendix §7.4, Table 5"}],"recommendation":"reject","confidential_remarks":"The algebraic error in Theorem 1 is independently verifiable and is not a presentation issue: the proposed optimal discriminator fails the first-order condition for the very objective introduced in the paper. Combined with the unquantified surrogate gap and the mismatch between the disjoint-support assumption and the experimental protocol, I do not see a local revision that would preserve the paper's central theoretical claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe short version: LLP-GAN is a plausible and useful extension of GAN-based semi-supervised learning to label-proportion problems, with public code and clear empirical gains over DLLP on standard benchmarks. The theory, though, is not as clean as advertised: the global optimality results hold for a Jensen lower-bound surrogate, not the loss actually minimized during training, and the disjoint-bag assumption in Definition 1 is violated by the random bags used in Section 4. Still, the core construction is worth taking seriously.\n\nWhat is genuinely new: transferring the SSL-GAN framework to LLP is natural but non-obvious, and the closed-form optimal discriminator in Eq. (9) — a density-weighted average of bag proportions — is a nice characterization. Theorem 2's claim that the generator's optimum is the average of bag marginals is elegant, though it inherits the surrogate issue. The empirical study is honest: they compare against DLLP, InvCal, and alter-∝SVM across four datasets and several bag sizes, report training time scaling, and the code is available. The conclusion lists limitations candidly, which I appreciate.\n\nWhere it gets soft. Remark 2 replaces the bag-level cross-entropy -CEL(p,p) with a Jensen lower bound, swapping the log of an average for an average of logs. Theorems 1 and 2 characterize the equilibrium of that surrogate, and no bound on the Jensen gap is supplied. When bag supports overlap — which is exactly what happens with random bag construction — the 'optimal' discriminator may not optimize the actual LLP objective. Lemma 1's proof also contains an inequality that is not justified, and the pointwise conclusion ~p_D*(y|x)=p_i(y) relies on the bag-specific marginal assumption that the experiments do not satisfy. The authors are upfront about some limitations, but the theoretical claims are oversold relative to what is proven.\n\nOne more practical gap: the comparison omits concurrent deep LLP work, notably Dulac-Arnold et al. (2019), which they cite as [10] and mention in passing. Their own DLLP baseline is weaker without data augmentation, so the headline numbers are less convincing than they could be.\n\nBottom line: this is a within-subfield contribution that deserves referee time, but it should go back for major revision. The theory either needs to be reworked to address the gap between surrogate and true objective, or reframed honestly as an analysis of the surrogate. The experiments should include concurrent methods. A careful reader will still learn something from the construction and the empirical behavior.","headline":"A useful LLP-GAN extension with solid empirical gains, but the global-optimality theory applies to a Jensen lower-bound surrogate, not the actual loss, and the disjoint-bag assumption clashes with the random-bag experiments.","tokens_in":13263,"tokens_out":2709,"would_cite":true,"duration_ms":23508,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A GAN whose optimal discriminator directly solves learning from label proportions.","keywords":["learning from label proportions","generative adversarial networks","weakly supervised learning","instance-level classification","proportion cross-entropy","multi-class classification","deep learning","feature matching"],"falsifier":"On a small two- or three-class dataset with known bag proportions, train LLP-GAN and then compute both the lower-bound objective and the true bag-level proportion cross-entropy for the trained discriminator on held-out bags. If the discriminator does not approximately maximize the true cross-entropy while maximizing the surrogate, the global optimality theorem does not carry over to the training objective. A more direct check is to verify whether the trained discriminator's bag-averaged posteriors match the true proportions on held-out bags; systematic mismatch would contradict Lemma 1's claim that the optimal posterior equals the bag prior at equilibrium.","tokens_in":12326,"feed_emoji":"🎯","tokens_out":6034,"duration_ms":54007,"temperature":0.7,"pith_summary":"Learning from label proportions asks for an instance-level classifier when training data comes only in bags, with each bag labeled by the fraction of its items in each class and no individual labels. This paper proposes LLP-GAN, an adversarial network in which a generator produces fake examples and a discriminator both separates real from fake and predicts the $K$ real classes. The paper claims that this game has a well-defined equilibrium: for a fixed generator the optimal discriminator has a closed form, and at the global optimum the generator density equals the average of the bag densities. If true, the discriminator itself is the classifier and LLP becomes a standard end-to-end deep learning problem, avoiding the distributional assumptions and scalability limits of earlier shallow LLP methods.","feed_headline":"Bag proportions alone train a GAN instance classifier","feed_subtitle":"The adversarial game's optimal discriminator directly outputs per-instance labels, with a proven global optimum.","key_machinery":"The load-bearing object is the $(K{+}1)$-way discriminator, which treats generated samples as a separate fake class and real samples as one of $K$ classes. The argument works by replacing the bag-level proportion cross-entropy with its Jensen lower bound (Eq. 6), which swaps the logarithm and the bag average; maximizing that lower bound is equivalent to minimizing the KL divergence between each bag's known proportion vector and the discriminator's mean posterior over that bag. This yields the closed-form optimal discriminator of Theorem 1 and, through Jensen-Shannon divergence, the global optimality condition of Theorem 2. In practice the generator is trained with feature matching rather than the pure adversarial loss, to avoid the known instability of a perfect generator.","core_discovery":"The central claim is that label-proportion supervision can be injected into a GAN through a proportion cross-entropy term, and the resulting adversarial objective has a solvable optimum. The paper proves that maximizing a Jensen lower bound of that term makes the optimal discriminator take the explicit form $P_D^*(y=k|x) = (\\sum_{i=1}^n p_i(k) p_i^d(x))/(\\sum_{i=1}^n p_i^d(x)+p_g(x))$ for the $K$ real classes; the normalized posterior $\\tilde{p}_{D^*}(y|x)$ is then a weighted average of the bag priors $p_i(y)$ with weights proportional to the bag densities at $x$. Theorem 2 states that the global minimum of the generator loss is reached if and only if $p_g = \\frac{1}{n}\\sum_i p_i^d$, recovering the usual GAN equilibrium in the one-bag case. Consequently the discriminator can be used directly as the instance-level classifier, and the final classification does not depend on the generator density, so the result also covers generators whose density is degenerate.","pith_inferences":["A direct testable consequence not explored in the paper: if the Jensen lower-bound gap is large, training on the surrogate could converge to a discriminator that is optimal for the surrogate but suboptimal for the true proportion cross-entropy; comparing the two objectives on a small dataset would quantify the gap.","The density-weighted aggregation formula implies that bags concentrating their mass at $x$ dominate the prediction there; for non-random bags built on sensitive features, this could introduce bias, a limitation the paper lists as future work.","The independence of the final classifier from $p_g$ suggests a two-stage design where the generator is only used to estimate the bag densities, after which the classifier could be solved as a weighted mixture estimation problem, potentially making the method robust to generator collapse."],"forward_implications":["The discriminator's output after training is the instance-level classifier, so no separate classifier or label-recovery postprocessing is needed.","LLP-GAN scales to large datasets and large bag sizes where the DLLP baseline diverges; for example, on CIFAR-10 at bag size 128 the reported error is 27.39% versus DLLP's 82.89%.","The global optimum pins down the generator's target density as the average of the bag densities, giving a clear training target and recovering the standard GAN result when there is one bag.","Because the optimal classifier aggregates bag priors with density weights, the method does not require bags to be i.i.d. samples from a single distribution; each bag can have its own marginal density."],"supporting_citations":[{"why":"Introduces the adversarial game formulation that LLP-GAN adapts.","marker":"[11]"},{"why":"Supplies the semi-supervised GAN template, the $K{+}1$ discriminator trick, and the feature matching objective.","marker":"[28]"},{"why":"Defines the deep LLP baseline DLLP that LLP-GAN compares against and extends.","marker":"[1]"},{"why":"Establishes the LLP problem and the mean-map estimation approach used as a baseline.","marker":"[23]"},{"why":"Provides the $\\propto$-SVM LLP solver used as a baseline in binary experiments.","marker":"[34]"},{"why":"Warns that a well-trained generator can degrade supervised information, motivating feature matching.","marker":"[8]"},{"why":"Identifies the density-degeneracy issue for generators, which Remark 3 addresses by removing $p_g$ from the classifier.","marker":"[2]"}],"fun_headline_variants":["Proportion-only GAN yields instance labels","Bag labels supervise GAN, output per-instance class","Proven optimum for GAN with label proportions","From bag proportions, GAN classifies individually"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof of global optimality applies to the Jensen lower-bound surrogate of the proportion cross-entropy, not to the true proportion loss actually minimized during training, and the paper never bounds the gap between the two.","fun_headline_variants_meta":{"raw":{"variants":["Proportion-only GAN yields instance labels","Bag labels supervise GAN, output per-instance class","Proven optimum for GAN with label proportions","From bag proportions, GAN classifies individually"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000299,"raw_usage":{"total_tokens":1698,"prompt_tokens":883,"completion_tokens":815,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":755}},"tokens_in":499,"tokens_out":815,"duration_ms":8283,"temperature":1.0,"reasoning_tokens":755,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:57:42.780483+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small two- or three-class dataset with known bag proportions, train LLP-GAN and then compute both the lower-bound objective and the true bag-level proportion cross-entropy for the trained discriminator on held-out bags. If the discriminator does not approximately maximize the true cross-entropy while maximizing the surrogate, the global optimality theorem does not carry over to the training objective. A more direct check is to verify whether the trained discriminator's bag-averaged posteriors match the true proportions on held-out bags; systematic mismatch would contradict Lemma 1's claim that the optimal posterior equals the bag prior at equilibrium.","supporting_citations":[{"cited_title":"Generative adversarial nets","cited_arxiv_id":null,"evidence_quote":"Introduces the adversarial game formulation that LLP-GAN adapts."},{"cited_title":"Improved techniques for training GANs","cited_arxiv_id":null,"evidence_quote":"Supplies the semi-supervised GAN template, the $K{+}1$ discriminator trick, and the feature matching objective."},{"cited_title":"Ardehaly and Aron Culotta","cited_arxiv_id":null,"evidence_quote":"Defines the deep LLP baseline DLLP that LLP-GAN compares against and extends."},{"cited_title":"Smola, Tiberio S","cited_arxiv_id":null,"evidence_quote":"Establishes the LLP problem and the mean-map estimation approach used as a baseline."},{"cited_title":"Yu, Dong Liu, Sanjiv Kumar, et al.∝-SVM for learning with label proportions","cited_arxiv_id":null,"evidence_quote":"Provides the $\\propto$-SVM LLP solver used as a baseline in binary experiments."},{"cited_title":"Good semi-supervised learning that requires a bad gan","cited_arxiv_id":null,"evidence_quote":"Warns that a well-trained generator can degrade supervised information, motivating feature matching."},{"cited_title":"Towards principled methods for training generative adver- sarial networks","cited_arxiv_id":null,"evidence_quote":"Identifies the density-degeneracy issue for generators, which Remark 3 addresses by removing $p_g$ from the classifier."}],"review_version":1}