Cascaded Transformer-Enhanced CRF for Nested NER Recognition.
A modular M.Tech research project developing a cascaded model to detect overlapping entities across domains, utilizing Transformer representations and CRF structured prediction.
M.Tech Level
Academic Rigor
Nested NER
Deep Learning
Multi-Domain
Cross-Transfer
Cascaded Prediction Layers
Explicit support for overlapping and nested entity spans.
Biomedical Corpus Schema & Nested Span Inspector
Real-time inspection of recursive XML entity structures from the GENIA corpus. Validates character offsets, multi-level parent-child boundaries, and prevents data degradation prior to Transformer tokenization.
<sentence id="genia-1029">
Activation of the
<cons lex="human_immunodeficiency_virus_type_1_enhancer" sem="G#DNA_domain_or_region">
<cons lex="human_immunodeficiency_virus_type_1" sem="G#virus">human immunodeficiency virus type 1</cons> enhancer
</cons>
by <cons lex="interleukin-1" sem="G#protein_molecule">interleukin-1</cons>.
</sentence>pytest tests/test_xml_parser.py –disable-warnings -v
| Hierarchy | Entity Type | Offset Range |
|---|---|---|
| Parent Span | DNA Region | [18:62] |
| Nested Child | Virus / Organism | [18:53] |
| Nesting Level | Depth = 2 (Parent contains Child) | |
Standard flat BIO taggers discard nested inner entities. The custom parser implemented in src/data/parser.py stores hierarchical trees in canonical JSON-L schemas, retaining exact character offsets without truncating overlapping spans.
Architecture Comparison
Evaluating structural capacity for overlapping named entities. We contrast standard flat classifiers against linear sequence models and the proposed cascaded formulation.
Standard token-level classification via linear classification head.
[char 0:18 -> Token OOM/Collision]Sequence Modeling Strategy
Single-label BIO sequence tagging (B-ENT, I-ENT, O)
- •Fast single-pass token inference
- •Low GPU memory footprint on Colab
- •Standard cross-entropy loss calculation
- •Arbitrary nested spans cannot be tagged simultaneously
- •Inner sub-entities are permanently overwritten by outer tokens
- •Flat BIO assumption breaks empirical GENIA benchmark
Sequential joint transition modeling with Viterbi decoding trellis.
[char 14:38 -> Trellis Level-1 only]Sequence Modeling Strategy
Structured transition matrix over flattened multi-class tags
- •Eliminates illegal transition sequences (e.g., I-tag following O)
- •Stronger sequence boundary coherence
- •Optimal global sequence decoding path
- •Standard linear CRF alone fails on arbitrary span overlaps
- •Exponential label-space growth if combining paired labels
- •No top-down feedback from outer entity spans to inner tokens
Hierarchical span refinement conditioning inner detection on outer context.
[char 14:38 -> char 22:30 | L1->L2]Sequence Modeling Strategy
Dual-tier cascaded emissions with structured CRF boundary filter
- •Outer predictions directly inform inner span detection heads
- •Viterbi CRF integration for both primary and nested spans
- •Explicit multi-domain conditioning vectors (GENIA + ACE)
- •Requires synchronized two-tier loss supervision during training
- •Higher compute budget per batch in resource-limited Colab
Research Compliance & Honest Metrics Reporting
Flat BIO sequence models systematically miscount nested annotations. To maintain scientific rigor under IEEE guidelines, evaluations on the GENIA biomedical dataset measure exact hierarchical span-and-type F1 rather than collapsed token accuracy.
Cascaded Transformer-Enhanced CRF
A modular, reproducible research pipeline designed for nested named entity recognition across multiple domains.
Modular Project Structure
Stage 1Establish a clean, modular Python repository with dedicated directories for data, models, training, and evaluation to ensure reproducibility.
GENIA Dataset Parsing
Stage 2Acquire the GENIA nested NER dataset, inspect XML schemas, and implement a parser to extract complete inner and outer entity spans.
Text & Span Alignment
Stage 3Normalize text, tokenize for Transformers, and convert nested spans into a format compatible with the proposed cascaded architecture.
Baseline Model Training
Stage 4Implement flat BERT, Transformer-CRF, and nested-NER baselines to establish performance benchmarks for the proposed model.
Cascaded Transformer-CRF
Stage 5Develop the cascaded architecture using Transformer encoders, domain-aware features, and CRF-based structured prediction for nested entities.
Empirical Results & Evaluation Metrics
Quantitative analysis of the Cascaded Transformer-Enhanced CRF architecture against flat and standard sequence baselines on the GENIA nested biomedical entity dataset.
Exact span-boundary and category match
Cascaded feedback loop precision
Batch size 32 with dynamic padding
With domain adapter projection
BIO sequence models force flat classification, discarding the nested protein span entirely. The cascaded layer reconstructs the hierarchy.
| Class Type | Tokens | Precision | Recall | F1 Score | Delta |
|---|---|---|---|---|---|
| DNA | 1058 | 84.3% | 82.9% | 83.6% | +12.4% |
| RNA | 118 | 79.1% | 76.4% | 77.7% | +15.1% |
| protein | 3412 | 88.2% | 86.9% | 87.5% | +9.2% |
| cell_type | 840 | 81.0% | 79.5% | 80.2% | +11.8% |
| cell_line | 495 | 78.4% | 74.2% | 76.2% | +14.6% |
| Model Setup | Outer F1 | Inner F1 | Overall |
|---|---|---|---|
1. Transformer Flat Baseline (No Cascade) Flat token classifier misses overlapping inner spans | 78.4% | 18.2% | 54.3% |
2. Transformer + Linear CRF CRF structure improves token transitions but collapses nesting | 80.1% | 25.6% | 58.2% |
3. Cascaded Model w/o Domain Adaptation Strong in-domain nested extraction on GENIA | 84.9% | 75.1% | 80.0% |
4. Proposed Model w/o CRF Trellis Demonstrates contribution of structured emission decode | 83.2% | 73.9% | 78.5% |
5. Complete Proposed Model (Cascaded + CRF + DA) Optimal balance on deep span recursion and boundary precision | 87.1% | 81.4% | 84.9% |
All training checkpoints, metric logs (JSON), and PyTorch model weights are reproducible on Google Colab GPU runtimes.
Modular Repository & Test Architecture
The project rejects single-notebook scripting in favor of decoupled, reusable Python modules. Every stage incorporates automated test validation, GPU memory bounds checking, and canonical YAML configuration to guarantee verifiable reproduction in Google Colab.
| Path | Stage | Functional Purpose | Implementation Detail |
|---|---|---|---|
| data/raw/ | Stage 2 | Raw Source Ingestion | GENIA XML source annotations and ACE corpus splits without modifications. |
| data/processed/ | Stage 3 | Canonical Offset Alignment | Normalized token streams with nested span character offsets and depth mappings. |
| src/data/ | Stage 2-3 | Parsing & Alignment | XML parsers, subword aligners, and span integrity validation matrices. |
| src/models/ | Stage 4-5 | Model Architecture | BERT encoders, cascaded feature extractors, and linear CRF sequence heads. |
| src/training/ | Stage 5-7 | Optimization & Scheduling | Loss computation, gradient clipping, early stopping, and GPU batch monitors. |
| src/evaluation/ | Stage 7 | Metric Computation | Strict boundary and type matching, inner/outer F1 metrics, and ablation logs. |
| src/inference/ | Stage 5-8 | Predictive Runtime | End-to-end span extraction returning hierarchical parent-child entity tuples. |
| outputs/checkpoints/ | Stage 5 | Saved Model Weights | Serialized PyTorch state dicts with configuration hashes and training steps. |
| tests/ | Stage 9 | Automated Suite | Pytest modules covering XML parsing, token alignments, and CRF emission shapes. |
| experiment_config.yaml | Stage 1 | Reproducibility Config | Hyperparameter seeds, encoder checkpoints, and cross-domain transfer flags. |
project_meta: name: "NERX-Cascaded-CRF" academic_unit: "Dayananda Sagar Univ" reproducibility_seed: 42 model_config: encoder: "microsoft/BiomedNLP-PubMedBERT" max_span_length: 12 cascade_levels: 2 use_crf_viterbi: true domain_embedding_dim: 64 training_hyperparameters: learning_rate: 2.0e-5 crf_learning_rate: 1.0e-3 batch_size: 16 gradient_accumulation_steps: 2 warmup_ratio: 0.1 max_epochs: 15 fp16_enabled: true
Checkpoint Organization
Every training cycle saves atomic weights to outputs/checkpoints/best_model.pt alongside the exact Git commit SHA and validation logs to prevent unverified metric reporting.
Reproducibility Protocols & Execution Specifications
Detailed step-by-step procedures for students and peer researchers cloning this repository.
Ready to execute the reproducible research pipeline?
Launch the verified Google Colab notebook preconfigured with GPU memory detection and GENIA XML parsers.
Model Inference Console
Test nested entity extraction in real-time.
Project Reproducibility
Technical guidance for the M.Tech project on Cascaded Transformer-Enhanced CRF for Multi-Domain Nested NER, covering project structure, data validation, model baselines, and IEEE paper requirements.
Project Modular Structure
Dataset Acquisition & Validation
Baseline & Proposed Models
IEEE Research Paper
Ready to Start Stage 1?
Begin by creating the project structure and inspecting the GENIA dataset schema.