BlossomPsy: A User-Centric AI System for Adaptive and Engaging MBTI Personality Assessments

Bingjia Huang

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

AI Chatbot Matches MBTI Scale With Higher User Engagement

2026-07-08 15:03 UTC pith:GAGYAPKW

load-bearing objection BlossomPsy integrates dialogue, photo-based questions, and a modified UCB bandit into an MBTI assessment pipeline. The system design is coherent and the paper is honest about its limitations, but the central consistency claim is undermined by a circular evaluation setup: 33 of 45 participants are LLMs prompted with known MBTI labels, and no human-only sub-analysis is reported. the 3 major comments →

arxiv 2607.06149 v2 pith:GAGYAPKW submitted 2026-07-07 cs.HC

BlossomPsy: A User-Centric AI System for Adaptive and Engaging MBTI Personality Assessments

Bingjia Huang This is my paper

classification cs.HC

keywords MBTIpersonality assessmentmulti-armed banditadaptive dialoguemulti-head classifierPID controllerhuman-computer interactionLLM-simulated users

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces BlossomPsy, an AI-driven personality assessment system that attempts to make the MBTI test more engaging without sacrificing accuracy. It replaces traditional static questionnaires with a multi-turn conversational agent powered by large language models, supplemented by photo-based questions when confidence is low. The core technical mechanism is a Multi-Head Classifier that predicts MBTI dimensions from text, combined with a modified Upper Confidence Bound algorithm that directs the conversation toward the dimensions the system is least certain about. A PID controller tunes the confidence parameters during training. The author evaluates the system against the Chinese MBTI-M scale using a mix of human volunteers and LLM-simulated participants, reporting moderate-to-strong agreement and higher user satisfaction compared to the traditional test.

Core claim

The central claim is that an adaptive, confidence-driven conversational system can produce MBTI predictions with comparable consistency to a standard questionnaire while delivering a more engaging user experience. The author reports Cohen kappa values ranging from 0.50 to 0.90 across the four MBTI dimensions when comparing BlossomPsy's output to MBTI-M, with users rating the system higher in interactivity and enjoyment. The paper frames these results as preliminary evidence of alignment rather than full psychometric validation.

What carries the argument

A Multi-Head Classifier with a shared RoBERTa encoder and five parallel classification heads (four binary MBTI dimension classifiers plus one 16-class classifier), paired with a modified Upper Confidence Bound algorithm that treats each MBTI dimension as a bandit arm and directs conversation toward the lowest-confidence dimension. A PID controller tunes the sigmoid enhancement function parameters during training. Photo-based questions serve as a fallback when confidence intervals overlap beyond a threshold.

Load-bearing premise

The consistency metrics are computed over a pool of 45 participants that mixes 12 humans with 33 LLM-simulated test-takers whose MBTI types are experimenter-assigned via prompting, which means the agreement numbers may be inflated by the system recovering labels it was indirectly given.

What would settle it

If human-only agreement metrics, computed separately from the LLM-simulated participants, show substantially lower Cohen kappa values than the pooled 0.50-0.90 range, the claim of comparable consistency with MBTI-M would be significantly weakened.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If adaptive dialogue systems can match questionnaire accuracy while improving engagement, personality assessment could become accessible to populations that find traditional tests tedious, such as adolescents and young adults.
The confidence-driven fallback mechanism, which inserts targeted visual questions only when text-based prediction is uncertain, offers a template for reducing assessment length without sacrificing coverage.
The framework is designed to be portable to other personality models such as the Big Five, requiring only replacement of binary classification heads with continuous regression heads, though this transfer has not been tested.
The photo-generation pipeline, which uses multiple LLMs with human supervision to create and validate visual question items, suggests a semi-automated approach to scale development that could lower the cost of creating new assessment instruments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The evaluation pools 12 human participants with 33 LLM-simulated participants whose MBTI types are assigned by prompting, meaning the agreement metrics may partially reflect the system's ability to recover labels that were used to construct the simulated test-takers rather than genuine predictive accuracy on humans.
If the S/N dimension is inherently harder to detect through text because it reflects abstract cognitive preferences, then conversational assessment systems may need fundamentally different question strategies or additional modalities for certain traits, rather than uniform adaptation across all dimensions.
The PID controller tunes only two scalar parameters, and the author notes grid search might achieve comparable results, suggesting the control-theoretic machinery may be more complex than the problem requires.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

BlossomPsy integrates dialogue, photo-based questions, and a modified UCB bandit into an MBTI assessment pipeline. The system design is coherent and the paper is honest about its limitations, but the central consistency claim is undermined by a circular evaluation setup: 33 of 45 participants are LLMs prompted with known MBTI labels, and no human-only sub-analysis is reported.

read the letter

The main thing to know: this paper has a genuine system-design contribution wrapped in an evaluation that can't support its headline consistency numbers. The circularity concern from the stress-test is real and lands cleanly — I checked Fig. A6 and the prompts are exactly as described. The LLM test-takers are told their MBTI type and instructed to embody it, then the system tries to recover that label. With 33 of 45 participants in this regime and no human-only breakdown of the Cohen's kappa values in Tab. 2, the pooled metrics (0.50–0.90) are not interpretable as evidence of consistency on real users. The ablation (Fig. 10) uses exclusively LLM-simulated participants, so the claimed contributions of PID, mUCB, and photo-based modules are also evaluated under this circular regime. That's the load-bearing problem. What's actually new and done well: the integration of a modified UCB bandit for confidence-driven dimension selection, combined with a photo-based fallback and a PID-tuned sigmoid enhancement, is a legitimate system design not present in PsyDI or PsychoGoat. The photo generation pipeline — interpreter, generator, monitor, with human supervision and iterative refinement — is a reasonable engineering contribution, and Tab. 3 shows human supervision meaningfully improves acceptance rate. The MHC fine-tuned on Personality Cafe achieves solid F1 scores (0.838–0.920), competitive with prior work in Tab. 5. The paper is consistently and appropriately framed as preliminary, and the limitations section (§5.4) is candid — the author acknowledges the small human sample, the possibility that PID adds no value over grid search, and the need for psychometric validation. Soft spots beyond the circularity issue: the human sample is 12 participants across 16 MBTI types (Tab. A4), which is too thin to support any dimension-level claim on real users. The PID controller tunes only two scalar parameters (α, β), and the paper itself notes simpler search strategies may match it. No code or data is released. The MHC without fine-tuning on Personality Cafe is near chance (Tab. 4, 0.488–0.514 F1), which raises questions about generalization to populations outside the training distribution. That said, the paper does not overclaim. It says 'preliminary evidence of alignment,' not validation. The system architecture is novel as an integration, the photo pipeline is a real contribution, and the honesty about limitations is genuine rather than performative. Who gets value: HCI researchers working on interactive assessment systems, and anyone interested in confidence-driven adaptive questioning pipelines. The paper deserves a serious referee who can push the author to report human-only metrics, expand the human sample, and release code. The core design is worth engaging with even though the current evidence is insufficient for the consistency claims as stated.

Referee Report

3 major / 9 minor

Summary. BlossomPsy is an interactive AI-driven MBTI assessment system that combines multi-turn dialogue, photo-based questions, a Multi-Head Classifier (MHC) with a modified UCB (mUCB) confidence mechanism, and PID-tuned sigmoid enhancement. The paper reports preliminary consistency results (Cohen's kappa 0.50–0.90 across MBTI dimensions) from 45 participants (12 humans + 33 LLM-simulated), user satisfaction ratings favoring BlossomPsy over MBTI-M, an ablation study, and MHC benchmark comparisons. The system is presented as a research prototype, and the authors explicitly frame results as preliminary evidence of alignment rather than full psychometric validation.

Significance. The paper presents a creative integration of multi-armed bandit confidence estimation, PID-controlled parameter tuning, and multi-modal (text + photo) interaction for personality assessment. The photo-based question generation pipeline with iterative LLM-and-human supervision is a novel contribution to interactive assessment design. The authors are commendably transparent about limitations: they explicitly state the system is a research prototype, acknowledge MBTI psychometric debates, and frame results as preliminary. The MHC benchmark comparisons (Table 5) and the ablation study structure provide useful baselines. However, the central consistency claim is substantially weakened by the evaluation methodology, particularly the pooling of human and LLM-simulated participants without separate reporting.

major comments (3)

§4.2.1–4.2.2, Table 2: The pooled 45-participant sample (12 humans + 33 LLMs) makes the reported Cohen's kappa values (0.50–0.90) difficult to interpret. The LLM-simulated participants are prompted with a known MBTI type (Fig. A6, left: 'You are role-playing as a person with the MBTI personality type {{mbti}}'), and the system then attempts to recover this label from the LLM's generated responses. This creates a partial circularity: the ground truth for 73% of the sample is the label assigned by the experimenters' own prompting. The photo-based evaluation is even more explicitly circular (Fig. A6, right: the LLM is told its type and asked to choose photos based on 'your {{pretend_mbti}} personality would most naturally prefer'). The paper acknowledges LLM construct validity concerns in §5.1 but does not report human-only consistency metrics. A human-only sub-analysis (even with n=12) is载
§4.3.2, Fig. 10 (Ablation Study): The ablation study uses exclusively LLM-simulated participants, meaning the claimed contributions of PID, mUCB, and photo-based modules are evaluated entirely under the circular ground-truth condition described above. The ablation results should be interpreted with this limitation clearly stated, or ideally supplemented with human data for at least the full-system condition. Without this, the component-level claims (e.g., 'PID and photo-based questions were associated with higher prediction accuracy') are not independently validated on real users.
§3.3, Eq. (4): The cumulative reward update ĥ_i(t) = [T_i(t-1) × ĥ_i(t-1) + mUCB_i(t)] / T_i(t) is recursive in mUCB_i(t), which itself depends on ĥ_i(t-1) via Eq. (2). This creates a self-referential loop that is not standard in UCB formulations. The authors should clarify whether this is intentional and how convergence is affected, or correct the formulation if this is a notational error.

minor comments (9)

§4.2.3, Fig. 8: It is unclear whether the user satisfaction ratings were collected from the 12 human participants only or from all 45 (including LLM-simulated). This should be specified explicitly.
§4.2.4, Table 3: The phrase 'Average Convergent Rate' is non-standard; consider 'Average Agreement Rate' or 'Average Convergence Rate' with a clear definition.
Table 4: MHC performance on Personality Cafe without fine-tuning (F1 ≈ 0.49–0.51) is near chance for binary classification, yet this is not discussed. The large improvement after fine-tuning (0.84–0.92) suggests the fine-tuning dataset may be small or the base model is not learning generalizable features. This warrants brief discussion.
§3.5.2: The simulated user noise injection (20% posts from other MBTI types) is described, but the sensitivity of results to this 20% rate is not analyzed. A brief note on robustness would strengthen the claim that this simulates real-world inconsistency.
§5.4: The authors acknowledge that simpler alternatives like grid search may achieve comparable results for PID parameter tuning. Given this, the motivation for PID over grid search should be strengthened, or the PID contribution should be framed more modestly.
Table A4: The distribution of test-takers across MBTI types is shown, but it is unclear which entries are human vs. LLM-simulated. This breakdown should be added to enable assessment of human-only type coverage.
§3.3, Eq. (1): The sigmoid enhancement function f(p_i) = 1/(1 + α·e^{-β·p_i}) is described as 'sigmoid-like' but differs from the standard sigmoid. The motivation for this specific form over a standard sigmoid should be stated more clearly.
Fig. A6: The prompts for LLM-simulated test-takers are provided in the appendix, which is good for transparency. However, the main text (§4.2.1) should briefly summarize the prompting approach and its implications for evaluation validity, rather than relegating this entirely to the appendix.
The paper would benefit from a brief note on whether the 13 text-based questions selected for photo conversion (§4.2.4) were reviewed by anyone with psychometric expertise, or whether the selection was purely based on the stated heuristic rules.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee raises three major points: (1) the pooling of human and LLM-simulated participants in the consistency evaluation creates interpretive difficulties, particularly partial circularity for the LLM-simulated subset; (2) the ablation study relies exclusively on LLM-simulated participants, so component-level claims are not independently validated on real users; and (3) the mUCB cumulative reward update in Eq. (4) appears self-referential because mUCB_i(t) depends on ĥ_i(t-1) via Eq. (2), creating a non-standard recursive loop. We address each point below.

read point-by-point responses

Referee: §4.2.1–4.2.2, Table 2: The pooled 45-participant sample (12 humans + 33 LLMs) makes the reported Cohen's kappa values (0.50–0.90) difficult to interpret. The LLM-simulated participants are prompted with a known MBTI type, creating a partial circularity. The paper does not report human-only consistency metrics. A human-only sub-analysis (even with n=12) is requested.

Authors: The referee is correct that pooling human and LLM-simulated participants without separate reporting makes the consistency metrics difficult to interpret, and that the LLM-simulated condition involves a degree of circularity: the ground-truth label for 73% of the sample is the MBTI type assigned via the role-playing prompt, and the system then attempts to recover that label from the LLM's generated responses. We agree that human-only metrics should be reported separately. We will add a human-only sub-analysis (n=12) to the revised manuscript, reporting per-dimension accuracy, F1, and Cohen's kappa for the 12 human participants alone, alongside the pooled results. We will also add explicit discussion of the circularity concern for the LLM-simulated subset, making clear that the LLM results should be interpreted as a pipeline stress-test (can the system recover a known type end-to-end?) rather than as independent psychometric evidence. The pooled metrics will be retained for completeness but clearly flagged as combining two evaluation modalities with different epistemic status. revision: yes
Referee: §4.3.2, Fig. 10 (Ablation Study): The ablation study uses exclusively LLM-simulated participants, meaning the claimed contributions of PID, mUCB, and photo-based modules are evaluated entirely under the circular ground-truth condition. Component-level claims are not independently validated on real users.

Authors: This is a fair and accurate observation. The ablation study as currently reported uses only LLM-simulated participants, so the component-level claims (e.g., that PID and photo-based questions are associated with higher prediction accuracy) are validated only under the simulated condition where ground truth is assigned by prompting. We will revise the manuscript in two ways. First, we will add an explicit caveat in §4.3.2 stating that the ablation results are obtained under the LLM-simulated condition and should be interpreted as a controlled pipeline test rather than as independent validation on real users. Second, we will run the full-system condition and at least one ablation condition (photo-based questions removed) on the 12 human participants and report the comparison in the revised manuscript. We acknowledge that n=12 provides limited statistical power, but it will at least provide an initial human-data check on whether the direction of the ablation effect is consistent with the simulated results. We will be transparent that this is preliminary rather than definitive. revision: yes
Referee: §3.3, Eq. (4): The cumulative reward update ĥ_i(t) = [T_i(t-1) × ĥ_i(t-1) + mUCB_i(t)] / T_i(t) is recursive in mUCB_i(t), which itself depends on ĥ_i(t-1) via Eq. (2). This creates a self-referential loop that is not standard in UCB formulations. Clarify whether this is intentional and how convergence is affected, or correct the formulation if this is a notational error.

Authors: We thank the referee for identifying this issue. Upon careful review, we confirm that Eq. (4) as written is a notational error. In a standard UCB formulation, the cumulative reward update should use the observed reward (the MHC's enhanced logit score for the selected dimension at round t), not the mUCB value itself. The mUCB value in Eq. (2) is an exploration-augmented upper bound used for arm selection, not the reward signal that should be accumulated into the running mean. The intended update should be: ĥ_i(t) = [T_i(t-1) × ĥ_i(t-1) + r_i(t)] / T_i(t), where r_i(t) is the enhanced logit score f(p_i) from Eq. (1) for the selected dimension at round t. This removes the self-referential loop. We will correct Eq. (4) accordingly, update the surrounding text to clarify the distinction between the reward signal and the UCB exploration bonus, and verify that the implementation matches the corrected formulation. We will also add a brief remark on convergence: since the reward r_i(t) is bounded (as the output of a sigmoid function) and the update is a standard running average, the cumulative reward estimate converges under the standard assumptions for UCB-style algorithms. revision: yes

Circularity Check

0 steps flagged

33 of 45 evaluation participants are LLM-simulated with ground-truth MBTI labels injected via prompt; the system then 'predicts' those same labels, and pooled consistency metrics (Tab. 2) conflate this circular subset with 12 human participants.

full rationale

The paper's central consistency claim (Cohen's kappa 0.50–0.90 in Tab. 2) is computed over all 45 participants, but 33 are LLM-simulated participants whose 'true' MBTI type is the label injected via the role-playing prompt (Fig. A6). The system then attempts to recover this label from the LLM's generated responses, making the ground truth for 73% of the sample circular by construction. The photo-based question evaluation is even more explicitly circular: the LLM is told its type and asked to choose photos based on 'your {{pretend_mbti}} personality would most naturally prefer.' The ablation study (Fig. 10) uses exclusively LLM-simulated participants, so the claimed contributions of PID, mUCB, and photo-based modules are also evaluated under this circular setup. The paper acknowledges LLM construct validity concerns in §5.1 but does not report human-only consistency metrics, making it impossible to assess how much of the reported agreement comes from the circular LLM subset versus genuine human performance. The 12 human participants provide some independent data, but the pooled metrics are dominated by the circular majority, and the paper does not separate them. This is partial circularity: the central claim is not entirely circular (12 humans contribute non-circular data), but the majority of the evidence is forced by construction. The MHC training on Personality Cafe (self-reported MBTI labels) and evaluation against MBTI-M is a separate concern about label quality but is not itself circular in the definitional sense, since the training labels and the evaluation reference scale are different instruments. No self-citation chain is load-bearing; the circularity is methodological, not citation-based. Score 6 reflects that one or more 'predictions' reduce by construction for the LLM subset, while the human subset retains independent content that the paper does not isolate or report separately.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 2 invented entities

The system introduces 6 free parameters (several hand-set), relies on 5 domain/ad-hoc assumptions (most critically that LLM-simulated participants approximate human behavior and that self-reported MBTI labels are valid training signal), and invents 2 entities (mUCB, photo-based items) that lack independent external validation. The PID controller is acknowledged by the authors as potentially unnecessary.

free parameters (6)

alpha (sigmoid enhancement) = converges to ~15.6 region (Fig. 11a)
Parameter in the sigmoid-like enhancement function f(p_i) = 1/(1+alpha*exp(-beta*p_i)), tuned by PID controller during training on simulated data.
beta (sigmoid enhancement) = determined by alpha via constraint f(0.5)=0.5
Second parameter of the enhancement function, linked to alpha by the constraint that the function passes through (0.5, 0.5).
overlap threshold = 0.6
Threshold for triggering photo-based questions when mUCB/mLCB overlap rate exceeds this value. Set by hand.
PID gains (Kp, Ki, Kd) = Kp=1, Ki=0.02, Kd=0.01
PID controller gains for tuning alpha and beta, specified in Algorithm 1.
photo agreement threshold = 0.66
Threshold for retaining photo-based questions based on agreement with text counterparts.
reward values (r_correct, r_incorrect) = r_correct=0.5, r_incorrect=0
Reward signal values used in PID training, specified in Algorithm 1.

axioms (5)

domain assumption MBTI dimensions can be reliably predicted from text-based conversational responses
The entire MHC+mUCB pipeline assumes that user text in multi-turn dialogue carries sufficient signal for MBTI prediction. The paper acknowledges S/N is hard to detect via text (§5.2) but the system architecture depends on this assumption globally.
domain assumption LLM-simulated participants exhibit stable, measurable personality patterns that approximate human test-taker behavior
Invoked in §4.2.1 to justify pooling 33 LLM-simulated participants with 12 human volunteers. The paper cites Serapio-García et al. [37] and Zheng et al. [36] but also acknowledges counter-evidence [33, 49, 50].
domain assumption Self-reported MBTI labels in the Personality Cafe dataset are valid ground truth for training
The MHC is trained on Personality Cafe posts with self-reported MBTI labels (§3.5.1). MBTIBench [32] is cited as addressing noisy self-reported labels, but the training still relies on this data.
ad hoc to paper PID feedback control is an appropriate mechanism for tuning confidence transformation parameters
The use of a PID controller (Algorithm 1) to tune alpha and beta is a design choice. The authors themselves note 'simpler search strategies may achieve similar results' (§4.3.3, §5.4).
domain assumption Photo-based questions probe the same personality construct as their text-based counterparts
The photo-based module assumes visual stimuli can substitute for text items in measuring MBTI dimensions. The 80% consistency threshold (§3.4) is used as evidence, but no psychometric validation of construct equivalence is provided.

invented entities (2)

mUCB (modified Upper Confidence Bound) no independent evidence
purpose: Adapts UCB bandit to MBTI dimension selection by using enhanced logits and overlap rate for confidence estimation
The modification is specific to this paper. The ablation study (Fig. 10) provides internal evidence of contribution, but no external benchmark or falsifiable prediction outside the system is provided.
Photo-based MBTI question items no independent evidence
purpose: Fallback evaluation mechanism for low-confidence dimensions, designed to increase engagement and provide additional personality signal
The photo items are generated by LLMs and validated only by internal consistency (agreement with text counterparts). No psychometric validation of construct validity is provided.

pith-pipeline@v1.1.0-glm · 20803 in / 3337 out tokens · 486438 ms · 2026-07-08T15:03:46.785408+00:00 · methodology

0 comments

read the original abstract

There has been growing public interest in understanding personality traits and emotional characteristics, as such knowledge helps individuals better accept themselves and manage negative emotions. While professional personality scales remain the standard tool for assessment, they are often perceived as tedious or inaccessible to the general public. AI-driven systems can make assessments more accessible, but it is difficult to balance user engagement with predictive consistency in existing works. We tackle this challenge by introducing BlossomPsy, a user-friendly AI-driven MBTI assessment system. MBTI, a widely recognized but psychometrically debated personality framework, serves as the foundation for many recent systems. BlossomPsy integrates multi-turn dialogue and photo-based questions to enhance user engagement while supporting confidence-aware predictions. By combining deep learning, multi-armed bandit algorithms, and control theory, the system dynamically adapts to users' responses. In particular, photo-based questions are designed to increase interactivity and provide additional user information, thereby improving prediction confidence. Experiments involving both human volunteers and large language models (LLMs) provide preliminary evidence that BlossomPsy can produce stable predictions, with higher reported user satisfaction compared to MBTI-M (Chinese version), while maintaining comparable consistency with the reference scale.