---
license: apache-2.0
base_model: openai-community/gpt2
pipeline_tag: token-classification
tags:
- gpt2
- token-classification
- nlp
- event-extraction
- eventx
- sequence-labeling
datasets:
- mdg-nlp/eventx-recognition-original
metrics:
- f1
---
# GPT-2 Fine-Tuned for Event Expression Extraction (EVENTX)
## Overview
This model is a fully fine-tuned version of `openai-community/gpt2` for **Event Expression Extraction (EVENTX)**.
The model identifies spans in natural language text that correspond to events, including actions, occurrences, processes, and states expressed in a sentence.
The task is formulated as token-level sequence labeling using BIO tags.
### Label Map
| ID | Label | Description |
|---|---|---|
| 0 | O | Token is outside an event expression |
| 1 | B-E | Beginning of an event expression |
| 2 | I-E | Inside an event expression |
---
## Task
**Event Expression Extraction (EVENTX)**
Given an input sentence, the model identifies spans corresponding to event expressions.
For example:
```text id="2nxbq8"
Input:
The company announced the acquisition on Monday.
Event expressions:
announced
acquisition
```
Each token is assigned one of three BIO labels:
- **O**: outside an event expression
- **B-E**: beginning of an event expression
- **I-E**: inside an event expression
Event expression extraction provides structured information about events mentioned in text and can support downstream temporal reasoning, event ordering, relation extraction, and timeline construction.
---
## Model
GPT-2 is a decoder-only transformer architecture. This model uses GPT-2 with a token-classification head and is fully fine-tuned for sentence-level event expression extraction.
The model is one of the fine-tuned pretrained language model baselines used to study multi-dimensional generalization in temporal extraction tasks.
---
## Dataset
**Dataset:** `mdg-nlp/eventx-recognition-original`
### Labels
```text id="i9llpo"
0 -> O
1 -> B-E
2 -> I-E
```
---
## Training Details
| Configuration | Value |
|---|---|
| Base Model | `openai-community/gpt2` |
| Task | Event Expression Extraction |
| Formulation | Token Classification |
| Label Scheme | BIO |
| Fine-tuning | Full fine-tuning |
| Epochs | 3 |
| Learning Rate | 2e-5 |
| Optimizer | AdamW |
| Weight Decay | 0.01 |
| Warmup Steps | 500 |
| Train Batch Size | 16 |
| Evaluation Batch Size | 32 |
| Max Sequence Length | 128 |
---
## Evaluation
All reported scores use **micro-F1**.
| Evaluation Setting | Micro-F1 |
|---|---:|
| Base | **0.84** |
| T2C | 0.18 |
| Adversarial (Adv) | 0.78 |
| Length | 0.85 |
| Vocabulary (Voc) | 0.84 |
| Adversarial Perturbation (Adv_P) | 0.50 |
The evaluation settings measure generalization across the original task, cross-domain data, adversarial examples, structural changes, vocabulary variation, and adversarial perturbations.
---
## Usage
```python id="fjbd21"
from transformers import AutoTokenizer, AutoModelForTokenClassification
model_id = "mdg-nlp/gpt-2-eventx-sentence-recognition"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)
text = "The company announced the acquisition on Monday."
inputs = tokenizer(text, return_tensors="pt", truncation=True)
outputs = model(**inputs)
predictions = outputs.logits.argmax(dim=-1)[0].tolist()
id2label = {
0: "O",
1: "B-E",
2: "I-E"
}
tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
for token, prediction in zip(tokens, predictions):
print(token, id2label[prediction])
```
---
## Limitations
- Performance may decrease under distribution shifts.
- Cross-domain event expressions may be more difficult to identify.
- Adversarial perturbations can affect extraction performance.
- Tokenization can affect predicted event boundaries.
- Longer or structurally different inputs may produce different behavior.
- Ambiguous or implicit events may not be identified correctly.
- The model extracts event expressions and does not directly perform event relation or temporal relation classification.
---
## Research Context
This model was developed as part of research on **multi-dimensional generalization in temporal extraction tasks**.
The broader work studies temporal and event expression extraction under different forms of distribution shift, including domain shift, adversarial perturbations, vocabulary variation, compositional changes, and structural changes.
---
## Contact
**Author:** Arnav Verma
**Affiliation:** New York University
---
## Citation
If you use this model in your research, please cite the associated work when available.