TFBS-Finder: Deep Learning-based Model with DNABERT and Convolutional Networks to Predict Transcription Factor Binding Sites

Daniele Santoni; Nimisha Ghosh; Pratik Dutta

REVIEW 2 major objections 6 minor 37 references

TFBS-Finder: Deep Learning-based Model with DNABERT and Convolutional Networks to Predict Transcription Factor Binding Sites

T0 review · 2 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TFBS-Finder claims state-of-the-art prediction of transcription factor binding sites by combining DNABERT's global sequence embeddings with two complementary attention modules that refine local features.

desk verdict Plausible incremental architecture; the headline superiority claim rests on imported baselines and a margin within noise, so it deserves peer review but not acceptance without a controlled comparison. read the letter →

arxiv 2502.01311 v1 pith:QCD7B7MU submitted 2025-02-03 cs.LG cs.AIq-bio.GN

Nimisha Ghosh , Pratik Dutta , Daniele Santoni This is my paper · ORCID

classification cs.LGcs.AIq-bio.GN

keywords transcriptionfactorbindingsitesDNABERTdeeplearningattentionmechanismconvolutionalneuralnetworkChIP-seqENCODEsequenceclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Transcription factor binding sites are short DNA regions where regulatory proteins attach, and predicting them from sequence alone is a key step in reading gene regulatory networks. This paper proposes TFBS-Finder, a deep learning model that combines a pre-trained DNA language model (DNABERT) with convolutional layers and two attention modules to reach an average accuracy of 0.930, PR-AUC of 0.961, and ROC-AUC of 0.961 across 165 ENCODE ChIP-seq datasets. The authors claim these results beat seven existing predictors, including BERT-TFBS, by margins of 7.9% in accuracy, 4.1% in PR-AUC, and 4.2% in ROC-AUC. If the result holds, it would mean that pairing a transformer-based sequence encoder with locally focused attention mechanisms is a practical recipe for TFBS prediction, and the public release of code and a trained model would let others test it directly.

What carries the argument

The model's load-bearing mechanism is the combination of DNABERT's bidirectional self-attention over k-mer tokens (capturing long-range dependencies, with k=5 chosen after trying 3-6) and two local attention modules: MCBAM, which applies spatial-then-channel attention to the CNN's feature map, and MSCA, which runs three parallel convolutions, concatenates them, and uses a pointwise convolution with sigmoid to re-weight the original features. The output module fuses the refined features from both branches via addition, pooling, and a two-layer MLP with softmax. The ablation table (Table I) is the evidence that each component contributes to the final average scores.

What would settle it

Re-run BERT-TFBS and TFBS-Finder on identical train/test partitions (e.g., the same positive/negative sequence files, same random seed, same k-mer tokenizer) and compute the average accuracy, PR-AUC, and ROC-AUC across the 165 datasets; if the gap narrows from 7.9%/4.1%/4.2% to near zero, the claimed superiority is an artifact of protocol mismatch.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a hybrid architecture—DNABERT for global, context-aware sequence embeddings; a CNN for local motif extraction; a modified convolutional block attention module (MCBAM) that applies spatial attention before channel attention; and a multi-scale convolutions with attention module (MSCA) that re-weights features across scales—predicts transcription factor binding sites more accurately than any of its seven compared baselines (accuracy 0.930, PR-AUC 0.961, ROC-AUC 0.961). The ablation study shows each module contributes: removing MSCA drops accuracy to 0.909, removing MCBAM drops it to 0.900, and reversing the attention order (CBAM-style) gives 0.927, slightly below the full model. Cross-cell line validation for CTCF yields ROC-AUC scores above 0.93 in every train/test cell-line pair, which the authors read as evidence of generalizability rather than cell-line-specific memorization.

Load-bearing premise

The comparison rests on the assumption that the reported numbers for the seven baseline models (including BERT-TFBS) were obtained under the same train/test data split, sequence preprocessing, and evaluation protocol as TFBS-Finder, even though the paper does not re-run those models or specify the exact splits and seeds.

Editorial extensions

If this is right

If correct, TFBS-Finder offers a ready-to-use predictor that can annotate candidate regulatory regions in new cell types without re-training, since the trained model is released.
The ablation results imply that for this task, spatial-before-channel attention is slightly better than the reverse order, and that MSCA only helps when combined with MCBAM, which could guide architecture choices in similar sequence-classification problems.
The cross-cell line results suggest binding-site models can be transferred across cell lines for at least CTCF, reducing the need for per-cell-line ChIP-seq experiments.
The 4-6% improvement over the previously best BERT-TFBS suggests that adding multi-scale attention to a transformer-based embedding is a meaningful gain over the best prior model, not a wholesale change in approach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors report k=5 as best; an untested but plausible extension is to let the model use multiple k-mer sizes simultaneously (e.g., an ensemble or multi-head tokenizer) and see whether the gains stack.
The cross-cell-line setup only covers CTCF; a natural test is to repeat it for TFs with more variable binding motifs (e.g., YY1) and to check whether the model's learned attention maps highlight known motif positions.
Since the paper does not compare against recent larger DNA foundation models beyond BERT-TFBS, the claimed state-of-the-art status may be contingent on the chosen baseline set; testing against those models directly would delineate the boundary of the claim.
The public code release enables a direct reproducibility check: if the supplied trained model reproduces the reported averages on the same datasets, the result is robust to implementation details.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

You should know two things up front. The architecture is a sensible engineering combination — DNABERT embeddings, a CNN, a CBAM with spatial-before-channel ordering, and a multi-scale convolution with attention — and the authors evaluate it on 165 ENCODE ChIP-seq datasets, with ablations, cross-cell-line checks, and a public code link. That is real work. But the paper's central claim — that TFBS-Finder beats BERT-TFBS by 7.9% in accuracy and ~4% in PR-AUC/ROC-AUC — rests on baseline numbers imported from [6] rather than re-run in a shared pipeline, and the margin over the closest ablation (Variant 5) is 0.003 in accuracy and a tie in the other two metrics.

What is genuinely new: the particular combination of modules and the reordered attention, both of which are described clearly enough to reproduce. The ablation study is useful: it shows DNABERT alone gets 0.918 accuracy and each added module moves things only a few thousandths. The cross-cell-line validation for CTCF is a nice generalization test and shows robust performance.

The soft spot is the comparison protocol. Section III.D says only 'the dataset is the same as considered in [6]'. No shared re-implementation, no exact train/test splits, no seeds, no negative-sampling details for the baselines. Given that the ablation margin is within run-to-run variation, the claimed superiority over BERT-TFBS could easily be an artifact of evaluation protocol rather than architecture. There are also no confidence intervals or paired tests across the 165 datasets, so even the internal comparisons lack significance support. Minor issues: Eq. (7) has unbalanced parentheses and Eq. (12) appears to have a typo in the conv numbering, but these don't affect the core logic.

The paper deserves a serious referee because the architecture is new, the evaluation breadth is real, and the code is promised. But the referee should ask for matched baseline re-implementation or at least error bars and per-dataset significance analysis before the claims are accepted.

Recommendation: engage with the work, but treat the headline superiority claim as unproven until a controlled comparison is done.

Referee Report

2 major / 6 minor

Summary. The paper proposes TFBS-Finder, a deep learning model that combines a pre-trained DNABERT embedding module with a CNN, a modified convolutional block attention module (MCBAM), a multi-scale convolution with attention module (MSCA), and an output module for transcription factor binding site prediction. The model is trained and tested on 165 ENCODE ChIP-seq datasets. The authors report average accuracy, PR-AUC, and ROC-AUC values of 0.930, 0.961, and 0.961, respectively, and claim superiority over seven existing predictors, including BERT-TFBS, DeepBind, and DanQ. They also perform an ablation study, cross-cell line validation for CTCF, and provide publicly available code and data.

Significance. If the reported comparison is reliable, the work offers a practically useful, incremental improvement in TFBS prediction on a widely used benchmark, and the public release of code and datasets is a clear strength. The architecture is not conceptually novel—it is a combination of DNABERT with attention modules—but the large-scale evaluation and reproducible implementation are valuable. However, the strength of the central claim depends on whether the baseline comparison is truly controlled, which the manuscript does not currently demonstrate. The ablation study also lacks statistical backing for its small performance differences. These issues are load-bearing because the paper's headline result is the claimed superiority over existing methods.

major comments (2)

[III.D, Table III] The central claim that TFBS-Finder outperforms BERT-TFBS by 7.9% accuracy, 4.1% PR-AUC, and 4.2% ROC-AUC is not supported by a controlled comparison. The manuscript only states that 'the dataset is the same as considered in [6]' and does not report whether the baseline models were re-run in the same codebase, what exact train/validation/test splits and random seeds were used, or how negative samples were generated for each model. Since the full model beats its closest ablation (Variant 5) by only 0.003 in accuracy and ties in PR-AUC and ROC-AUC (Table I), the difference between TFBS-Finder and BERT-TFBS could be partly or wholly due to protocol differences rather than architectural improvements. Please provide a matched-protocol re-implementation of all baselines, or at least per-dataset paired comparisons with confidence intervals or significance tests (e.g., Wilcoxon signed-rank test) across the 165 datasets.
[III.B, Table I] The ablation study reports differences between variants without any measure of variability. The proposed model outperforms Variant 5 by only 0.003 in accuracy and is exactly tied on PR-AUC and ROC-AUC, so the claim that 'TFBS-Finder works slightly better' is not statistically supported. Run-to-run variation from random initialization and data shuffling could be larger than these margins. Please report standard deviations or confidence intervals across repeated runs or perform paired statistical tests across the 165 datasets to demonstrate that the spatial-before-channel order in MCBAM and the overall module combination yield a reliable improvement.

minor comments (6)

[II.B.4 and III.B] There is an inconsistency regarding the use of dilated convolutions: Section III.B states that the MSCA module captures broader local context 'using dilated convolutions,' but the architecture description in Section II.B.4 mentions only three parallel convolutions (Conv4,1, Conv4,2, Conv4,3) with concatenation and a pointwise convolution, with no dilation mentioned. Please clarify which description is correct and make the equations match the implementation.
[II.B.5, Eq. (12)] Equation (12) appears to contain a typographical error: the two branches use Conv 7 and Conv 6 inconsistently (the first branch is Conv7(M3)+Conv6(M4) while the second is Conv6(M3)+Conv7(M4)), whereas the prose says both M3 and M4 are passed through Conv 6 and then combined. Please correct the equation or the accompanying text.
[II.B.2, Eq. (7)] Equation (7) is missing a closing parenthesis in the sigmoid expression and is therefore unreadable as printed. Please check the parentheses and ensure the equation is syntactically correct.
[III.D] The manuscript should state explicitly where the baseline metric values come from (e.g., whether they are copied from the original publications or from [6]) and whether any additional preprocessing or postprocessing was applied. This is important for reproducibility even if a full re-implementation is not feasible.
[II.C] The key hyperparameters (k-mer size k, dropout values, batch size, number of epochs, early stopping patience) are only referenced to Supplementary Tables S2 and S3. Including at least the final values in the main text would make the paper more self-contained, especially since the supplementary tables are not visible in the arXiv version.
[Throughout] There are several minor typographical errors: 'Enclyopedia' in Section II.A should be 'Encyclopedia', 'verge' in the caption of Table III should be 'average', and 'T ranscription' in the page header should be 'Transcription'. These should be corrected in a final proofread.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity; the central result is an empirical benchmark comparison that does not reduce to its inputs.

full rationale

TFBS-Finder is an empirical deep-learning benchmark paper. The central claim (Table III) is that the proposed architecture reaches accuracy, PR-AUC, and ROC-AUC of 0.930, 0.961, and 0.961 on the 165 ENCODE ChIP-seq datasets, compared with numbers for BERT-TFBS and other baselines. This is not a derivation: the reported values are actual prediction outcomes of a trained classifier on benchmark data, not algebraic consequences of the input features or of a fitted parameter renamed as a prediction. The ablation study (Table I) compares architectural variants under the same protocol; the closeness of Variant 5 to TFBS-Finder is a statistical observation, not a reduction or a renamed fit. The only reference to the authors' own prior work is [23] in the introduction, and it is descriptive rather than load-bearing for the claimed superiority; no uniqueness theorem, ansatz, or defining relation from [23] is invoked. The statement "the dataset is the same as considered in [6]" imports baseline metrics from an external source, which may raise protocol-reproducibility concerns, but importing an external benchmark result is not circularity because TFBS-Finder's prediction is not defined in terms of those baselines. No self-definitional equation, no fitted input called a prediction, and no self-citation chain that forces the conclusion were found. The paper's limitations, if any, concern experimental reproducibility rather than circular reasoning, so the appropriate score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The model introduces no new biological entities or forces; the only contributions are architectural choices and hyperparameters tuned on the benchmark. The assumptions are standard for this style of applied deep-learning paper, with the notable exception that baseline comparability is assumed rather than demonstrated.

free parameters (6)

k-mer size (k) = 5
Section III.A states 'we have used different values of k (3, 4, 5 and 6) and observed that with k=5, we get the best results.' This is a hyperparameter fitted to the benchmark.
Dropout values = 0.2 and 0.3
Section II.C reports dropout values 'set to 0.2 and 0.3' to prevent overfitting, chosen by experimentation.
Batch size = 64
Section II.C sets batch size eta to 64.
Number of epochs = 15
Section II.C states training is performed for 15 epochs.
Early stopping patience = 2
Section III.A says training stops if PR-AUC does not improve for 2 consecutive validation runs.
Spatial-before-channel attention order = spatial then channel
Section III.B reports this ordering 'provides a slightly better result' than the channel-spatial order, so it is selected based on validation performance.

assumptions (4)

standard math The transformer self-attention formulation of BERT is a valid encoder for DNA sequences.
Section II.B.1 relies on equations (1)-(3) from reference [20] without additional justification.
domain assumption Pretrained DNABERT embeddings capture useful nucleotide context for TFBS prediction.
The method assumes that DNABERT's pretrained k-mer embeddings are transferable to the TFBS classification task, as stated in Section II.B.1.
domain assumption The Zeng et al. dataset, with positives from ChIP-seq peaks and negatives from dinucleotide-preserving shuffling, is a valid benchmark for TFBS prediction.
Section II.A describes the data source and negative sampling strategy; the validity of shuffled negatives is assumed.
domain assumption Baseline performance numbers from BERT-TFBS [6] and other cited papers are directly comparable to the authors' own runs.
Section III.D states the dataset is the same as in [6] but does not re-implement baselines; comparability of splits and preprocessing is assumed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TFBS-Finder: Deep Learning-based Model with DNABERT and Convolutional Networks to Predict Transcription Factor Binding Sites." pith.science (2026). https://pith.science/paper/QCD7B7MU

@misc{pith2026250201311,
  author       = {Pith},
  title        = {Pith review of: TFBS-Finder: Deep Learning-based Model with DNABERT and Convolutional Networks to Predict Transcription Factor Binding Sites},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QCD7B7MU}},
  note         = {Machine review of arXiv:2502.01311}
}

read the original abstract

Transcription factors are proteins that regulate the expression of genes by binding to specific genomic regions known as Transcription Factor Binding Sites (TFBSs), typically located in the promoter regions of those genes. Accurate prediction of these binding sites is essential for understanding the complex gene regulatory networks underlying various cellular functions. In this regard, many deep learning models have been developed for such prediction, but there is still scope of improvement. In this work, we have developed a deep learning model which uses pre-trained DNABERT, a Convolutional Neural Network (CNN) module, a Modified Convolutional Block Attention Module (MCBAM), a Multi-Scale Convolutions with Attention (MSCA) module and an output module. The pre-trained DNABERT is used for sequence embedding, thereby capturing the long-term dependencies in the DNA sequences while the CNN, MCBAM and MSCA modules are useful in extracting higher-order local features. TFBS-Finder is trained and tested on 165 ENCODE ChIP-seq datasets. We have also performed ablation studies as well as cross-cell line validations and comparisons with other models. The experimental results show the superiority of the proposed method in predicting TFBSs compared to the existing methodologies. The codes and the relevant datasets are publicly available at https://github.com/NimishaGhosh/TFBS-Finder/.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 35 canonical work pages

[6]

BERT-TFBS: a novel bert- based model for predicting transcription factor binding sites by transfer learning,

K. Wang, X. Zeng, J. Zhou, and et al., “BERT-TFBS: a novel bert- based model for predicting transcription factor binding sites by transfer learning,” Briefings in Bioinformatics , vol. 25, no. 3, p. bbae195, 05 2024

work page 2024
[1]

Sequences are divided into tokens of size k and provided as initial input to DNABERT

DNABERT: DNABERT is a pre-trained bidirectional encoder to enocode DNA sequences using k-mer technique. Sequences are divided into tokens of size k and provided as initial input to DNABERT. Token and positional embeddings are applied to such tokens to form the input matrix M. This input matrix is then passed through N sequential encoders to get the embedd...

work page
[2]

CNN Module: A single convolutional layer is used in this module to extract high-order local features from the embed- ded matrix M1. This module consists of one convolutional block with convolutional operation Conv 1, batch normali- sation (BN), Gaussian error linear unit (GELU) activation function and dropout operation. This operations can be given as: M2...

work page
[3]

As given in [26], the order of the two submodules affects the overall performance and in their work they have considered Channel-Spatial module

Modified Convolutional Block Attention Module: Spatial and channel attention blocks [26] are used in this module to enhance important local features obtained from the previous module. As given in [26], the order of the two submodules affects the overall performance and in their work they have considered Channel-Spatial module. However, in our work (as rep...

work page
[4]

As shown in Figure 1, three separate convolutions Conv 4,1, Conv 4,2 and Conv 4,3 are applied to each input channel of the feature matrix M2

Multi-Scale Convolution with Attention Module: This mod- ule utilises multi-scale convolutions with attention to further enhance the local features captured by the feature extraction module. As shown in Figure 1, three separate convolutions Conv 4,1, Conv 4,2 and Conv 4,3 are applied to each input channel of the feature matrix M2. The results of such con-...

work page 2015
[5]

The aforementioned operations can be expressed as: MS = Sigmoid(Conv 2(M axpool(M2), Avgpool(M2))) (5) M′ 2 = MS ⊙ M2 (6) Once M ′ 2 is constructed, channel attention submodule is applied on M ′ 2 where each of its channel is subjected to global maxpooling and global average pooling separately. The resultant features are then individually passed through t...

work page
[7]

This has the effect of parallel attention [27] in the output module, thereby exploiting both MCBAM and MSCA modules

Output Module: The output block considers the feature matrices M3 and M4 respectively, obtained from MCBAM and MSCA modules. This has the effect of parallel attention [27] in the output module, thereby exploiting both MCBAM and MSCA modules. The matrices obtained from these modules are passed separately through convolutional operations Conv 6 and the resu...

work page 2015
[8]

Assessing computational tools for the discovery of transcription factor binding sites,

M. Tompa, N. Li, T. L. Bailey, and et al., “Assessing computational tools for the discovery of transcription factor binding sites,” Nature Biotechnology, vol. 23, p. 137–144, 2005

work page 2005

Show all 37 references

[9]

Tfbstools: an r/bioconductor package for transcription factor binding site analysis,

G. Tan and B. Lenhard, “Tfbstools: an r/bioconductor package for transcription factor binding site analysis,”Bioinformatics, vol. 32, no. 10, pp. 1555–1556, 2016

2016
[10]

A review of dna-binding proteins prediction methods,

K. Qu, L. Wei, and Q. Zou, “A review of dna-binding proteins prediction methods,” Current Bioinformatics,, vol. 14, no. 3, pp. 246–254, 2019

2019
[11]

models etc. Although traditional machine learning algo- rithms provide competitive results, they are dependent on other methods for feature extraction and they also do not work well with large-scale datasets. To address these challenges, recently many deep learning algorithms ...

2025 arXiv
[12]

Transcription factors: An overview,

D. S. Latchman, “Transcription factors: An overview,” The International Journal of Biochemistry & Cell Biology, vol. 29, no. 12, pp. 1305–1312, 1997

1997
[13]

Too many transcription factors: positive and negative inter- actions,

M. Karin, “Too many transcription factors: positive and negative inter- actions,” The New biologist , vol. 2, no. 2, pp. 126–131, 1990

1990
[14]

ChIPBase v3.0: the encyclope- dia of transcriptional regulations of non-coding rnas and protein-coding genes,

J. Huang, W. Zheng, P. Zhang, and et al., “ChIPBase v3.0: the encyclope- dia of transcriptional regulations of non-coding rnas and protein-coding genes,” Nucleic Acids Research, vol. 51, no. D1, pp. D46–D56, 11 2022

2022
[15]

DNA motif elucidation using belief propagation,

K.-C. Wong, T.-K. Chan, C. Peng, and et al., “DNA motif elucidation using belief propagation,” Nucleic Acids Research , vol. 41, no. 16, pp. e153–e153, 2013

2013
[16]

A Biophysical Approach to Transcription Factor Binding Site Discovery,

M. Djordjevic, A. M. Sengupta, and B. I. Shraiman, “A Biophysical Approach to Transcription Factor Binding Site Discovery,” Genome Research, vol. 13, no. 11, p. 2381–2390, 2003

2003
[17]

Identification of yeast transcriptional regula- tion networks using multivariate random forests,

Y . Xiao and M. R. Segal, “Identification of yeast transcriptional regula- tion networks using multivariate random forests,” PLOS Computational Biology, vol. 5, no. 6, pp. 1–18, 06 2009

2009
[18]

A flexible integrative approach based on random forest improves prediction of transcription factor binding sites,

B. Hooghe, S. Broos, F. v. Roy, and et al., “A flexible integrative approach based on random forest improves prediction of transcription factor binding sites,” Nucleic Acids Research, vol. 40, no. 14, pp. e106– e106, 04 2012

2012
[19]

Predicting effects of noncoding variants with deep learning–based sequence model,

J. Zhou and O. G. Troyanskaya, “Predicting effects of noncoding variants with deep learning–based sequence model,” Nature Methods, vol. 12, p. 931–934, 2015

2015
[20]

Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning,

B. Alipanahi, A. Delong, M. T. Weirauch, and et al., “Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning,” Nature Biotechnology, vol. 33, p. 831–838, 2015

2015
[21]

DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences,

D. Quang and X. Xie, “DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences,” Nucleic Acids Research, vol. 44, no. 11, pp. e107–e107, 04 2016

2016
[22]

DeepSite: bidirectional LSTM and CNN models for predicting DNA–protein binding,

Y . Zhang, S. Qiao, S. Ji, and et al., “DeepSite: bidirectional LSTM and CNN models for predicting DNA–protein binding,” International Journal of Machine Learning and Cybernetics volume , vol. 11, p. 841–851, 2020

2020
[23]

Cooperation of local features and global representations by a dual-branch network for transcription factor binding sites prediction,

Y . Yu, P. Ding, H. Gao, and et al., “Cooperation of local features and global representations by a dual-branch network for transcription factor binding sites prediction,” Briefings in Bioinformatics , vol. 24, no. 2, p. bbad036, 02 2023

2023
[24]

A novel convolution attention model for predicting transcription factor binding sites by combination of sequence and shape,

Y . Zhang, Z. Wang, Y . Zeng, and et al., “A novel convolution attention model for predicting transcription factor binding sites by combination of sequence and shape,” Briefings in Bioinformatics , vol. 23, no. 1, p. bbab525, 12 2021

2021
[25]

Deepstf: predicting transcription factor binding sites by interpretable deep neural networks combining sequence and shape,

P. Ding, Y . Wang, X. Zhang, and et al., “Deepstf: predicting transcription factor binding sites by interpretable deep neural networks combining sequence and shape,” Briefings in Bioinformatics , vol. 24, no. 4, p. bbad231, 06 2023

2023
[26]

SAResNet: self-attention residual network for predicting DNA-protein binding,

L.-C. Shen, Y . Liu, J. Song, and et al., “SAResNet: self-attention residual network for predicting DNA-protein binding,” Briefings in Bioinformatics, vol. 22, no. 5, p. bbab101, 04 2021

2021
[27]

Bert: Pre-training of deep bidirectional transformers for language understanding,

J. Devlin, M.-W. Chang, K. Lee, and et al., “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2019. [Online]. Available: https://arxiv.org/abs/1810.04805

2019 arXiv
[28]

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome,

Y . Ji, Z. Zhou, H. Liu, and et al., “DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome,” Bioinformatics, vol. 37, no. 15, pp. 2112–2120, 02 2021

2021
[29]

ProteinBERT: a universal deep-learning model of protein sequence and function,

N. Brandes, D. Ofer, Y . Peleg, and et al., “ProteinBERT: a universal deep-learning model of protein sequence and function,” Bioinformatics, vol. 38, no. 8, pp. 2102–2110, 02 2022

2022
[30]

Predicting transcription factor binding sites with deep learning,

N. Ghosh, D. Santoni, I. Saha, and et al., “Predicting transcription factor binding sites with deep learning,” International Journal of Molecular Sciences, vol. 25, no. 9, 2024

2024
[31]

An integrated encyclopedia of dna elements in the human genome,

T. E. P. Consortium, “An integrated encyclopedia of dna elements in the human genome,” Nature, vol. 489, pp. 57–74, 2012. GHOSH et al.: BARE DEMO OF IEEETRAN.CLS FOR IEEE JOURNALS 9 TFBS-FinderBERT-TFBS DSAC D-SSCA DLBSS CRPTS DeepBind DanQ 0.5 0.6 0.7 0.8 0.9 1.0 Accuracy TFB...

2012
[32]

Convolutional neural net- work architectures for predicting dna–protein binding,

H. Zeng, M. D. Edwards, G. Liu, and et al., “Convolutional neural net- work architectures for predicting dna–protein binding,” Bioinformatics, vol. 32, no. 12, p. i121–i127, 2016

2016
[33]

CBAM: Convolutional block attention module,

S. Woo, J. Park, J.-Y . Lee, and et al., “CBAM: Convolutional block attention module,” in Computer Vision – ECCV 2018 , 2018, pp. 3–19

2018
[34]

Integrating multiple visual attention mecha- nisms in deep neural networks,

F. Martinez and Y . Zhao, “Integrating multiple visual attention mecha- nisms in deep neural networks,” in 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC) , 2023, pp. 1191– 1196

2023
[35]

Predicting in-vitro transcription factor binding sites using dna sequence + shape,

Q. Zhang, Z. Shen, and D.-S. Huang, “Predicting in-vitro transcription factor binding sites using dna sequence + shape,” IEEE/ACM Transac- tions on Computational Biology and Bioinformatics , vol. 18, no. 2, pp. 667–676, 2021

2021
[36]

Predicting transcription factor binding sites using dna shape features based on shared hybrid deep learning architecture,

S. Wang, Q. Zhang, Z. Shen, and et al., “Predicting transcription factor binding sites using dna shape features based on shared hybrid deep learning architecture,” Molecular Therapy - Nucleic Acids , vol. 24, pp. 154–163, 2021

2021
[37]

Dnabert-2: Efficient foundation model and benchmark for multi-species genome,

Z. Zhou, Y . Ji, W. Li, and et al., “Dnabert-2: Efficient foundation model and benchmark for multi-species genome,” 2024. [Online]. Available: https://arxiv.org/abs/2306.15006

2024 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [6]

BERT-TFBS: a novel bert- based model for predicting transcription factor binding sites by transfer learning,

K. Wang, X. Zeng, J. Zhou, and et al., “BERT-TFBS: a novel bert- based model for predicting transcription factor binding sites by transfer learning,” Briefings in Bioinformatics , vol. 25, no. 3, p. bbae195, 05 2024

work page 2024

[2] [1]

Sequences are divided into tokens of size k and provided as initial input to DNABERT

DNABERT: DNABERT is a pre-trained bidirectional encoder to enocode DNA sequences using k-mer technique. Sequences are divided into tokens of size k and provided as initial input to DNABERT. Token and positional embeddings are applied to such tokens to form the input matrix M. This input matrix is then passed through N sequential encoders to get the embedd...

work page

[3] [2]

CNN Module: A single convolutional layer is used in this module to extract high-order local features from the embed- ded matrix M1. This module consists of one convolutional block with convolutional operation Conv 1, batch normali- sation (BN), Gaussian error linear unit (GELU) activation function and dropout operation. This operations can be given as: M2...

work page

[4] [3]

As given in [26], the order of the two submodules affects the overall performance and in their work they have considered Channel-Spatial module

Modified Convolutional Block Attention Module: Spatial and channel attention blocks [26] are used in this module to enhance important local features obtained from the previous module. As given in [26], the order of the two submodules affects the overall performance and in their work they have considered Channel-Spatial module. However, in our work (as rep...

work page

[5] [4]

As shown in Figure 1, three separate convolutions Conv 4,1, Conv 4,2 and Conv 4,3 are applied to each input channel of the feature matrix M2

Multi-Scale Convolution with Attention Module: This mod- ule utilises multi-scale convolutions with attention to further enhance the local features captured by the feature extraction module. As shown in Figure 1, three separate convolutions Conv 4,1, Conv 4,2 and Conv 4,3 are applied to each input channel of the feature matrix M2. The results of such con-...

work page 2015

[6] [5]

The aforementioned operations can be expressed as: MS = Sigmoid(Conv 2(M axpool(M2), Avgpool(M2))) (5) M′ 2 = MS ⊙ M2 (6) Once M ′ 2 is constructed, channel attention submodule is applied on M ′ 2 where each of its channel is subjected to global maxpooling and global average pooling separately. The resultant features are then individually passed through t...

work page

[7] [7]

This has the effect of parallel attention [27] in the output module, thereby exploiting both MCBAM and MSCA modules

Output Module: The output block considers the feature matrices M3 and M4 respectively, obtained from MCBAM and MSCA modules. This has the effect of parallel attention [27] in the output module, thereby exploiting both MCBAM and MSCA modules. The matrices obtained from these modules are passed separately through convolutional operations Conv 6 and the resu...

work page 2015

[8] [8]

Assessing computational tools for the discovery of transcription factor binding sites,

M. Tompa, N. Li, T. L. Bailey, and et al., “Assessing computational tools for the discovery of transcription factor binding sites,” Nature Biotechnology, vol. 23, p. 137–144, 2005

work page 2005