Pith. sign in

REVIEW 3 major objections 6 minor 40 references

LLMs can score meeting effectiveness segment by segment

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-05 18:26 UTC pith:G2W7DBW7

load-bearing objection New dataset and task formulation for fine-grained meeting effectiveness evaluation; the automatic pipeline works well under ideal conditions but degrades sharply with predicted segmentation. the 3 major comments →

arxiv 2604.17260 v2 pith:G2W7DBW7 submitted 2026-04-19 cs.CL

Rethinking Meeting Effectiveness: A Benchmark and Framework for Temporal Fine-grained Automatic Meeting Effectiveness Evaluation

Yihang Li , Chenhui Chu This is my paper
classification cs.CL
keywords meeting effectivenessLLM-as-a-judgetopic segmentationautomatic evaluationmulti-party dialoguemeta-evaluation datasetobjective achievementtemporal fine-grained analysis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes that meeting effectiveness can be measured automatically and at fine temporal granularity by dividing a meeting into topical segments and having a large language model score each segment's contribution to the meeting's overall objectives relative to the time it consumed. The authors define effectiveness as objective achievement divided by time cost, moving away from single post-hoc satisfaction surveys toward a continuous, segment-level assessment that captures where a meeting was productive and where it stalled. To support this, they built a dataset of 2,459 human-annotated segments from 130 meetings and developed a three-stage LLM pipeline: classify the meeting's objectives, segment the transcript by topic, and score each segment. The central claim is that LLM-generated segment scores correlate strongly with expert human ratings when given ground-truth segment boundaries (Spearman rho approximately 0.64), and that the approach generalizes across structured business meetings and unstructured discussions, with end-to-end performance from raw speech remaining comparable to text-based input.

Core claim

The paper establishes that the LLM-as-a-judge paradigm, when applied to temporal segments of a meeting transcript and grounded in a formal effectiveness criterion (objective achievement per unit time), produces segment-level scores that approximate human expert judgments with moderate-to-strong correlation. The key mechanism is a three-step pipeline: multi-label classification of meeting objectives from a validated 19-category taxonomy, fine-grained topic segmentation of the transcript, and chain-of-thought-based effectiveness scoring of each segment against the full set of meeting objectives. A segmentation alignment procedure using duration-weighted score averaging handles mismatches in 7.

What carries the argument

The central machinery is a three-stage LLM pipeline: (1) meeting objective classification using a structured three-round prompting strategy over a 19-category taxonomy, constrained to at most three objectives per meeting; (2) fine-grained topic segmentation that produces continuous, minimal topical units with LLM-generated labels and descriptions; (3) segment effectiveness evaluation using a G-Eval-inspired chain-of-thought form-filling approach with a sliding context window, where the model scores each segment on a 1-5 rubric defined by objective contribution and time efficiency. A duration-weighted segmentation alignment procedure maps predicted scores onto ground-truth boundaries to allow

Load-bearing premise

The framework assumes that a fixed set of 2-4 manually pre-defined objectives per meeting type adequately captures the common purpose against which all segment effectiveness should be measured, and that these objectives can be reliably synthesized from meeting content after the fact.

What would settle it

If the pre-defined meeting objectives systematically fail to capture what participants actually cared about, then both human annotations and LLM scores would be measuring effectiveness against the wrong target, and the reported correlations would reflect shared misalignment rather than genuine evaluation quality.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Organizations could deploy real-time meeting dashboards that flag low-effectiveness segments as they occur, enabling in-meeting interventions rather than post-hoc diagnosis.
  • The segment-level annotation methodology (2,459 segments from 130 meetings) substantially multiplies the data available per meeting compared to single-score approaches, potentially enabling training of specialized lightweight models for effectiveness prediction without relying on large LLMs.
  • The theoretical upper-bound analysis reveals that segmentation accuracy is a bottleneck comparable to scoring accuracy, directing future work toward joint optimization of segmentation and scoring rather than treating them as independent subtasks.
  • The framework's robustness to ASR errors (end-to-end correlation rho approximately 0.22 vs. 0.24 from text segmentation) suggests that LLM-based effectiveness evaluation degrades gracefully under speech recognition noise, making deployment on raw audio feasible.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the pre-defined objectives used for annotation are incomplete or mis-specified, both the human ground truth and the LLM scores may converge on measuring the wrong target, and the reported human-LLM correlation would not detect this failure because both sides share the same blind spot.
  • The finding that longer segments receive higher scores could indicate either that substantive discussions are genuinely longer or that annotators and LLMs conflate duration with productivity, which would bias the effectiveness criterion toward rewarding sustained talk regardless of output.
  • Extending the framework to meetings with emergent or contested objectives (where participants disagree on what the meeting was for) would test whether the single-set-of-objectives assumption holds when the common purpose is itself ambiguous.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces a new paradigm for evaluating meeting effectiveness at a fine-grained temporal level, defining effectiveness as the rate of objective achievement over time. It contributes the AMI-ME dataset (2,459 annotated segments from 130 AMI Corpus meetings) and an LLM-based automatic evaluation framework comprising objective classification, topic segmentation, and segment-level effectiveness scoring. The framework is benchmarked across multiple LLMs and evaluated end-to-end from raw speech. Under ideal conditions (ground-truth segmentation and objectives), the best LLMs achieve Spearman correlation of approximately 0.64 with human judgments; under the fully automatic pipeline, correlation drops to approximately 0.22.

Significance. The paper addresses a genuine gap: existing meeting effectiveness evaluation relies on coarse post-hoc surveys, and no prior work applies the LLM-as-a-judge paradigm to fine-grained meeting analysis. The AMI-ME dataset, with professional annotation and ICC reliability of 0.82-0.88, is a solid contribution. The upper bound analysis in Table 2, which decomposes the loss between segmentation mismatch and scoring degradation, is a valuable diagnostic tool. The end-to-end pipeline from raw speech is a practical and falsifiable evaluation setup. Code and dataset release is promised.

major comments (3)
  1. §7.4, Table 2: The central claim of the paper is automatic evaluation of meeting effectiveness. The headline result of Spearman ρ≈0.64 (Table 1) is achieved only with ground-truth segmentation and ground-truth objectives. The fully automatic pipeline achieves ρ≈0.22 (Table 2, '+ From Segmentation'). The paper's abstract states 'Our results validate the framework's effectiveness,' and §8 claims 'LLMs can achieve strong correlation with human judgments,' but the automatic result that actually corresponds to the paper's central claim is ρ≈0.22, not 0.64. The abstract and conclusion do not clearly distinguish the ideal-condition result from the automatic-pipeline result. This framing risks misleading readers about the framework's practical capability. The authors should revise the abstract, introduction, and conclusion to clearly state that the strong correlation holds only under idealized (
  2. §7.4, Table 2: The upper bound analysis reveals a critical finding that is underemphasized. For Qwen3-32B, the theoretical upper bound given predicted segmentation is ρ=0.6417, yet the measured correlation is only 0.2256. This gap of approximately 0.42 means the LLM's scoring itself degrades severely when segment boundaries differ from ground truth — the model scores well on ground-truth segments (0.6445) but poorly on predicted segments (0.2256), despite the upper bound showing that good scoring on those same predicted segments could achieve ~0.64. The paper acknowledges this in §7.4 ('the structural discrepancy... inherently penalizes the evaluation metric') but frames the loss as primarily a segmentation alignment issue. The data show that scoring fragility to boundary placement is the dominant factor. The authors should explicitly characterize this as a scoring sensitivity problem (i
  3. §5.2 and Appendix B.3: The paper defines meeting objectives as 'goals synthesized from the meeting's content upon its conclusion' (§3), but the annotation protocol uses manually pre-defined objectives (2-4 per meeting type) that are specific to the AMI Corpus and distinct from the 19-objective taxonomy used by the automatic framework (§6.1, Appendix B.3). This creates a mismatch: human annotators evaluate against AMI-specific objectives while the LLM evaluates against the general 19-category taxonomy. The paper does not report the objective classification accuracy in the main text (only in Appendix E: Micro-F1 of 0.81-0.86), nor does it analyze how objective misclassification propagates to scoring accuracy. Given that objectives are the reference against which all effectiveness is measured, this mismatch should be discussed and its impact quantified or bounded.
minor comments (6)
  1. §7.5: The WER of 31.95% is quite high. The claim that 'the effectiveness evaluation component is relatively robust to speech transcription errors' is supported only by the comparison between '+ From Segmentation' (0.2256) and '+ From Speech' (0.2180) for Qwen3, but for GPT-4o the drop is from 0.2360 to 0.2006, which is a 15% relative decrease. This should be acknowledged as a non-trivial degradation.
  2. Table 1: The dagger and double-dagger significance markers are defined as comparisons against Qwen3-32B (non-reasoning) and GPT-4o, but it is unclear why these two models are chosen as baselines for significance testing. A brief justification would help.
  3. §5.1: The reference-based segmentation method uses Gemini-2.5-Pro, but the comparison in Appendix B.2 is based on only 5 meetings with 24 variations. This is a small sample for model selection; a note acknowledging this limitation would be appropriate.
  4. Figure 2(c): The confusion matrix is described as being for 'topic segmentation, viewed as a binary classification task of identifying boundaries,' but it is unclear whether this is for the reference-based segmentation or for a specific LLM's segmentation. Clarification needed.
  5. §6.2: For reasoning LLMs, the paper samples 5 times and averages. It would be useful to report the variance across these samples to assess stability.
  6. The abstract states '2,459 human-annotated segments from 130 AMI Corpus meetings' but does not mention the number of annotators or the annotation process. A brief mention would strengthen the abstract.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. All three major comments identify legitimate issues that we will address in revision. Below we respond point by point.

read point-by-point responses

  1. Referee: §7.4, Table 2: The central claim of the paper is automatic evaluation of meeting effectiveness. The headline result of Spearman ρ≈0.64 (Table 1) is achieved only with ground-truth segmentation and ground-truth objectives. The fully automatic pipeline achieves ρ≈0.22 (Table 2, '+ From Segmentation'). The paper's abstract states 'Our results validate the framework's effectiveness,' and §8 claims 'LLMs can achieve strong correlation with human judgments,' but the automatic result that actually corresponds to the paper's central claim is ρ≈0.22, not 0.64. The abstract and conclusion do not clearly distinguish the ideal-condition result from the automatic-pipeline result. This framing risks misleading readers about the framework's practical capability. The authors should revise the abstract, introduction, and conclusion to clearly state that the strong correlation holds only under idealized (

    Authors: The referee is correct. The current abstract and conclusion do not adequately distinguish the ideal-condition result (ρ≈0.64 with ground-truth segmentation and objectives) from the fully automatic pipeline result (ρ≈0.22). This is a genuine framing problem that we will fix. In the revised manuscript, we will: (1) state both numbers explicitly in the abstract, noting that ρ≈0.64 is achieved under oracle conditions and ρ≈0.22 under the fully automatic pipeline; (2) revise the introduction to set up both the upper-bound and end-to-end results as distinct findings; and (3) rewrite the conclusion to accurately characterize the framework's current practical capability, acknowledging that the gap between idealized and automatic performance is a key challenge for future work. We agree that the phrase 'validate the framework's effectiveness' is too strong given the automatic-pipeline numbers and will replace it with a more precise statement. revision: yes

  2. Referee: §7.4, Table 2: The upper bound analysis reveals a critical finding that is underemphasized. For Qwen3-32B, the theoretical upper bound given predicted segmentation is ρ=0.6417, yet the measured correlation is only 0.2256. This gap of approximately 0.42 means the LLM's scoring itself degrades severely when segment boundaries differ from ground truth — the model scores well on ground-truth segments (0.6445) but poorly on predicted segments (0.2256), despite the upper bound showing that good scoring on those same predicted segments could achieve ~0.64. The paper acknowledges this in §7.4 ('the structural discrepancy... inherently penalizes the evaluation metric') but frames the loss as primarily a segmentation alignment issue. The data show that scoring fragility to boundary placement is the dominant factor. The authors should explicitly characterize this as a scoring sensitivity problem (i

    Authors: The referee's analysis is accurate and we agree that our current framing understates the scoring sensitivity problem. The data are clear: the theoretical upper bound given predicted segmentation (0.6417 for Qwen3-32B) is close to the ground-truth-segment performance (0.6445), which means the segmentation mismatch itself is not the primary bottleneck. The dominant factor is that the LLM's scoring degrades dramatically when segment boundaries differ from those it was effectively calibrated on — a 0.42 gap between what is achievable on the predicted segments and what the LLM actually achieves. In the revision, we will: (1) explicitly recharacterize this as a scoring sensitivity problem rather than primarily a segmentation alignment issue; (2) add a sentence in §7.4 stating that the gap between the upper bound and measured correlation indicates that LLM scoring is fragile to boundary placement, and that improving segmentation alone will not close the gap; and (3) discuss this as a key finding with implications for future work (e.g., training or prompting strategies that make scoring more robust to boundary variation). revision: yes

  3. Referee: §5.2 and Appendix B.3: The paper defines meeting objectives as 'goals synthesized from the meeting's content upon its conclusion' (§3), but the annotation protocol uses manually pre-defined objectives (2-4 per meeting type) that are specific to the AMI Corpus and distinct from the 19-objective taxonomy used by the automatic framework (§6.1, Appendix B.3). This creates a mismatch: human annotators evaluate against AMI-specific objectives while the LLM evaluates against the general 19-category taxonomy. The paper does not report the objective classification accuracy in the main text (only in Appendix E: Micro-F1 of 0.81-0.86), nor does it analyze how objective misclassification propagates to scoring accuracy. Given that objectives are the reference against which all effectiveness is measured, this mismatch should be discussed and its impact quantified or bounded.

    Authors: The referee correctly identifies a mismatch between the annotation-time objectives (AMI-specific, manually summarized, 2-4 per meeting type) and the automatic framework's objectives (the 19-category taxonomy from Standaert et al., 2016). We agree this should be discussed more prominently and its impact analyzed. We will make the following changes: (1) Add a paragraph in the main text (§6.1 or §7.2) that explicitly acknowledges the mismatch, explains the rationale (the 19-category taxonomy was chosen for generalizability beyond AMI, while the annotation-specific objectives were tailored for annotator clarity), and describes the mapping procedure used for evaluation. (2) Report the objective classification Micro-F1 (0.81-0.86) in the main text rather than only in the appendix. (3) Add analysis of how objective misclassification propagates to scoring: the ablation in Appendix E (Figure 5) already shows that predicted objectives yield lower but still reasonable correlation compared to ground-truth objectives (e.g., for Qwen3, the gap between GT-objectives and predicted-objectives conditions is modest relative to the overall performance), which provides a partial bound on the impact. We will make this analysis more explicit in the main text. We acknowledge that a fully controlled propagation analysis (e.g., per-objective error analysis) would strengthen the paper, and we will add what we can within the revision scope, but a comprehensive per-category error propagation study may remain as future work. revision: partial

Circularity Check

1 steps flagged

No significant circularity; one minor self-referential dependency in objective ground-truth construction

specific steps
  1. self definitional [Section 5.2 and Appendix D (Experiment Settings)]
    "The ground truth labels were derived from the manually summarized objectives created during the human annotation phase. As these objectives do not directly align with the predefined 19-category classification schema, we established a mapping from each ground truth objective to a set of relevant categories."

    The ground-truth labels for evaluating the LLM's objective classification step are derived from the same manually summarized objectives that were provided to human annotators. This creates a mild self-referential loop: the evaluation target for the classification component is constructed from the annotation process's own input materials, not from an independently derived source. However, this is a minor issue because (1) the classification step is upstream of the main effectiveness scoring claim, (2) the ablation study (Appendix E) shows the framework works with predicted objectives (not just ground-truth), and (3) the central correlation results (Table 1) measure LLM scores against independent human effectiveness ratings, not against the objective labels themselves. The circularity is ina

full rationale

The paper's central claim—that LLM-as-a-judge segment-level effectiveness scores correlate with human judgments—is not circular. Human annotations are collected from professional annotators with quality control (ICC 0.82-0.88), and LLM scores are generated from a separate pipeline. The correlation in Table 1 (rho ~0.64) measures LLM output against independently collected human ratings. The one minor self-referential dependency is that the ground-truth labels for the objective classification sub-task are derived from the same manually summarized objectives used in the annotation interface, but this sub-task is upstream of the main effectiveness scoring claim, and the ablation study confirms the framework functions with predicted objectives. No step in the main derivation chain reduces to its inputs by construction. The paper is largely self-contained against its external benchmark (human annotations).

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical entities, particles, forces, or mathematical objects. The AMI-ME dataset and the evaluation framework are methodological contributions, not ontological ones. The 19 meeting objectives are adopted from Standaert et al. (2016), not invented. The per-meeting-type objective sets are manually summarized from existing meeting content, not postulated entities.

free parameters (4)
  • Maximum objectives per meeting (3) = 3
    Set by manual inspection of AMI meeting types (§6.1). This cap constrains the objective classification output and affects downstream effectiveness scoring.
  • Context window size (1) = 1
    Selected as optimal from ablation (Appendix E). Affects LLM effectiveness scoring by controlling how much surrounding transcript context the model sees.
  • Number of annotation samples for reasoning LLMs (5) = 5
    Number of samples averaged for reasoning LLM scores (§6.2). Chosen without stated justification for this specific value.
  • Per-meeting-type objective sets (2-4 objectives each) = 2-4 per type
    Manually summarized objectives for each of 6 AMI meeting types (Appendix B.3). These define the ground truth against which all effectiveness is measured.
axioms (5)
  • domain assumption Meeting effectiveness equals objective achievement divided by time cost
    Stated in §3 as the foundational criterion. This is a modeling choice that equates 'effectiveness' with 'efficiency' and assumes objective achievement is the primary outcome of interest, excluding participant well-being and relational outcomes.
  • domain assumption Meeting objectives can be reliably synthesized from meeting content upon conclusion
    Stated in §3: 'we define the meeting objectives as the goals synthesized from the meeting's content upon its conclusion.' This assumes emergent objectives are identifiable and stable, but the annotation protocol uses pre-defined per-type objectives instead.
  • domain assumption Human-annotated effectiveness scores have a linear relationship with objective achievement
    Acknowledged in Limitations: 'Our definition of effectiveness and the subsequent segmentation alignment method implicitly assume that the human-annotated scores have a linear relationship with the degree of objective achievement.' The duration-weighted averaging in Appendix A depends on this.
  • domain assumption The AMI Corpus simulated meetings are representative enough for framework validation
    The entire dataset is built on AMI Corpus meetings. The paper acknowledges in Limitations that these 'are simulated business meetings that may not fully capture the higher stakes, implicit power hierarchies, organizational politics, and culturally contingent social norms of real-world interactions.'
  • domain assumption LLM token probability weighted summation produces meaningful continuous effectiveness scores
    §6.2 states that for non-reasoning LLMs, 'we use the token probability weighted summation of the output score as the final score.' This assumes the probability distribution over discrete score tokens is a meaningful proxy for continuous effectiveness, which is not independently validated.

pith-pipeline@v1.1.0-glm · 23980 in / 3522 out tokens · 197072 ms · 2026-07-05T18:26:14.744055+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Rethinking Meeting Effectiveness: A Benchmark and Framework for Temporal Fine-grained Automatic Meeting Effectiveness Evaluation." pith.science (2026). https://pith.science/paper/G2W7DBW7

@misc{pith2026260417260,
  author       = {Pith},
  title        = {Pith review of: Rethinking Meeting Effectiveness: A Benchmark and Framework for Temporal Fine-grained Automatic Meeting Effectiveness Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G2W7DBW7}},
  note         = {Machine review of arXiv:2604.17260}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Evaluating meeting effectiveness is crucial for improving organizational productivity. Current approaches rely on post-hoc surveys that yield a single coarse-grained score for an entire meeting. The reliance on manual assessment is inherently limited in scalability, cost, and reproducibility. Moreover, a single score fails to capture the dynamic nature of collaborative discussions. We propose a new paradigm for evaluating meeting effectiveness centered on novel criteria and temporal fine-grained approach. We define effectiveness as the rate of objective achievement over time and assess it for individual topical segments within a meeting. To support this task, we introduce the AMI Meeting Effectiveness (AMI-ME) dataset, a new meta-evaluation dataset containing 2,459 human-annotated segments from 130 AMI Corpus meetings. We also develop an automatic effectiveness evaluation framework that uses a Large Language Model (LLM) as a judge to score each segment's effectiveness relative to the overall meeting objectives. Through substantial experiments, we establish a comprehensive benchmark for this new task and evaluate the framework's generalizability across distinct meeting types, ranging from business scenarios to unstructured discussions. Furthermore, we benchmark end-to-end performance starting from raw speech to measure the capabilities of a complete system. Our results validate the framework's effectiveness and provide strong baselines to facilitate future research in meeting analysis and multi-party dialogue. Our dataset and code will be publicly available. The AMI-ME dataset and the Automatic Evaluation Framework are available at: this URL.

Figures

Figures reproduced from arXiv: 2604.17260 by Chenhui Chu, Yihang Li.

Figure 1
Figure 1. Figure 1: The paradigm of meeting effectiveness evalu [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Statistics of the AMI-ME dataset. (a) Distribution of segment count per meeting. (b) Distribution of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The automatic evaluation framework. mentation incorporated 1,668 of the 2,109 original boundaries and introduced 661 new ones, omitting many original boundaries to ensure continuity. 5.2 Human Annotations for Effectiveness After segmentation, we collected human annota￾tions for segment effectiveness through a rigorous quality control process. Given the complexity of the task and the quality differences bet… view at source ↗
Figure 4
Figure 4. Figure 4: The annotation interface. the meeting content fully, corpora from specialized domains like research or politics present a signif￾icant challenge due to the extensive background knowledge required. Therefore, we chose the AMI Corpus (Carletta et al., 2005), which is centered around business scenarios. The AMI Corpus is a multimodal dataset com￾prising 100 hours of meeting recordings. It is en￾riched with a … view at source ↗
Figure 5
Figure 5. Figure 5: Ablation studies of the context window size and the meeting objectives. (a) Experiments on Llama3.3- [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Relationship between segment scores and duration. (a) Human annotation. (b) Prediction of Qwen3-32B [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The Spearman correlation coefficient between [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Illustration of how segmentation granularity [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · 5 internal anchors

  1. [1]

    Statistical models for text segmentation. Mach. Learn., 34(1–3):177–210. Manik Bhandari, Pranav Narayan Gour, Atabak Ash- faq, Pengfei Liu, and Graham Neubig. 2020. Re- evaluating evaluation in text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9347–9359, Online. Association for Com...

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  2. [2]

    Comfeel: Productivity is a matter of the senses too. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 4(4). Ross Cutler, Yasaman Hosseinkashi, Jamie Pool, Senja Filipi, Robert Aichner, Yuan Tu, and Johannes Gehrke. 2021. Meeting effectiveness and inclu- siveness in remote collaboration. Proc. ACM Hum.-Comput. Interact., 5(CSCW1). DeepSeek-AI, Daya G...

    work page 2021

  3. [3]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning. Preprint, arXiv:2501.12948. Mingqi Gao, Xinyu Hu, Xunjian Yin, Jie Ruan, Xiao Pu, and Xiaojun Wan. 2025. LLM-based NLG evalu- ation: Current status and challenges. Computational Linguistics, 51:661–687. Boni García, Micael Gallego, Francisco Gortázar, and Antonia Bertoli...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  4. [4]

    The Llama 3 Herd of Models

    Analysis of Small Groups, pages 349–367. Dan Gillick and Yang Liu. 2010. Non-expert eval- uation of summarization systems is risky. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pages 148–151, Los Angeles. As- sociation for Computational Linguistics. Aaron Grattafiori, Abhimanyu Dubey, Ab...

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  5. [5]

    In Proceedings of the 31st International Conference on Computational Linguistics, pages 5027–5039, Abu Dhabi, UAE

    Evaluating open-source ASR systems: Per- formance across diverse audio conditions and er- ror correction methods. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5027–5039, Abu Dhabi, UAE. As- sociation for Computational Linguistics. A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gel- bart, N. Morgan, B. Peskin, T. Pf...

    work page 2003

  6. [6]

    An- nenberg School of Communications, University of Southern California

    A profile of meetings in corporate America: Results of the 3M meeting effectiveness study. An- nenberg School of Communications, University of Southern California. Andrew C. Morris, Viktoria Maier, and Phil D. Green

    work page

  7. [7]

    GPT-4o System Card

    From wer and ril to mer and wil: improved evaluation measures for connected speech recogni- tion. In Interspeech. Gabriel Murray and Catharine Oertel. 2018. Pre- dicting group performance in task-based interac- tion. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, ICMI ’18, page 14–20, New York, NY , USA. Association for...

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  8. [8]

    Unsupervised Topic Segmentation of Meetings with BERT Embeddings

    A nonverbal behavior approach to identify emergent leaders in small groups. IEEE Transactions on Multimedia, 14(3):816–832. Alessandro Solbiati, Kevin Heffernan, Georgios Damaskinos, Shivani Poddar, Shubham Modi, and Jacques Cali. 2021. Unsupervised topic segmenta- tion of meetings with bert embeddings. Preprint, arXiv:2106.12978. Willem Standaert, Steve ...

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  9. [9]

    Participant Introductions The project manager initiates a round of introductions where each team member states their name and role in the project. ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������� �������������������������������������...

    work page

  10. [10]

    Get acquainted to team members

    Effectively share information about the project 2. Get acquainted to team members

    work page

  11. [11]

    Learn to use drawing tools 4. Generate good ideas on remote control None of them Effectiveness: Ineffective Marginally Effective Moderately Effective Highly Effective Exceptionally Effective 🎯 Meeting Objectives

    work page

  12. [12]

    Effectively share information about the project

    work page

  13. [13]

    Get acquainted to team members

    work page

  14. [14]

    Learn to use drawing tools

    work page

  15. [15]

    Generate good ideas on remote control 📑 Agenda Summary Opening Acquaintance Tool training Peoject plan Discussion Closing 📋 Meeting Agenda

    work page

  16. [16]

    Project Goals: The primary objective is to design a new remote control that is original, trendy, and user- friendly

    Kick-off and Project Overview (Topics 1 - 2) Introductions: Team members introduced themselves and their roles: Laura (Project Manager), David (Industrial Designer), Andrew (Marketing), and Craig (User Interface). Project Goals: The primary objective is to design a new remote control that is original, trendy, and user- friendly. Design Process: The projec...

    work page

  17. [17]

    Figure 4: The annotation interface

    Team Icebreaker: Favorite Animal Drawings (Topics 3 - 6) As a warm-up activity, each team member drew their favorite animal on the whiteboard and described its characteristics. Figure 4: The annotation interface. the meeting content fully, corpora from specialized domains like research or politics present a signif- icant challenge due to the extensive bac...

    work page 2005

  18. [18]

    None of them

    and Gemini-2.5-Pro (Comanici et al., 2025). A comparative analysis was conducted on five ran- domly selected meetings. Taking Qwen3’s output as a baseline, we identified 24 variations (merges, splits, or boundary shifts) in Gemini-2.5-Pro’s seg- mentation. A review of these variations showed that Gemini-2.5-Pro’s output was superior in 14 cases, Qwen3’s w...

    work page 2025

  19. [19]

    Exchange/share opinions or views on a topic or issue

    work page

  20. [20]

    Give or receive orders

    work page

  21. [21]

    Find a solution to a problem that has arisen

    work page

  22. [22]

    Generate ideas on products, projects or initiatives

    work page

  23. [23]

    Generate buy-in or consensus on an idea

    work page

  24. [24]

    Resolve conflicts and disagreements within a group

    work page

  25. [25]

    Build trust and relationships with one or more individuals

    work page

  26. [26]

    Maintain relationships with one or more other people and stay in touch

    work page

  27. [27]

    Negotiate or bargain on a deal or contract

    work page

  28. [28]

    Routine exchange of information

    work page

  29. [29]

    Non-routine exchange of information

    work page

  30. [30]

    Communicate positive or negative feelings or emotions on a topic or issue

    work page

  31. [31]

    Show personal concern about or interest in a particular issue or situation

    work page

  32. [32]

    Assert and/or reinforce your authority, status, position to your team or others

    work page

  33. [33]

    Give or receive feedback

    work page

  34. [34]

    Assemble a team and/or motivate teamwork on a project

    work page

  35. [35]

    Clarify a concept, issue or idea

    work page

  36. [36]

    Round 3 - Final Selection: From remaining objectives, select up to 3 PRIMARY objectives with strongest evidence

    Exchange confidential, private or sensitive information The core context of the three-step meeting ob- jective classification prompt is shown as follows: Prompt Three-Round Selection Process: Round 1 - Identify potentially relevant objectives with their original ID numbers (1-19) Round 2 - Detailed Analysis: Examine evidence for each candidate objective, ...

    work page

  37. [37]

    Ensure each segment represents a coherent topic discussion with clear boundaries for optimal topic segmentation

    Divide the transcript into distinct segments based on topic changes. Ensure each segment represents a coherent topic discussion with clear boundaries for optimal topic segmentation

    work page

  38. [38]

    Make the segmentation as fine-grained as possible, identifying even subtle topic shifts, while maintaining topic coherence within each segment

    work page

  39. [39]

    - ‘end_id‘: The ID of the last utterance of the segment

    For each segment, provide: - ‘start_id‘: The ID of the first utterance of the segment. - ‘end_id‘: The ID of the last utterance of the segment. - ‘topic‘: A concise phrase describing the main topic. - ‘description‘: A one-sentence summary of the segment content

    work page

  40. [40]

    Generate good ideas on remote control

    Critical Check for Completeness and Continuity: - **No Gaps**: The ‘start_id‘ ID of any segment (except the first) must immediately follow the ‘end_id‘ ID of the preceding segment. For example, if segment N ends at ID 15, segment N+1 must start at ID 16. - **Full Coverage**: All utterances from the first utterance ID provided in the transcript to the very...

    work page 2024