scripts/train.py#

import argparse from datasets import load_dataset from transformers import AutoTokenizer, AutoModelForQuestionAnswering, TrainingArguments, Trainer

def parse_args(): parser = argparse.ArgumentParser(description=“Fine-tune a QA model.”) parser.add_argument(“—model_name”, type=str, default=“bert-base-uncased”, help=“Pretrained model name”) parser.add_argument(“—train_file”, type=str, required=True) parser.add_argument(“—val_file”, type=str, required=True) parser.add_argument(“—epochs”, type=int, default=3) parser.add_argument(“—batch_size”, type=int, default=8) parser.add_argument(“—lr”, type=float, default=3e-5) parser.add_argument(“—output_dir”, type=str, default=“models/checkpoints”) return parser.parse_args()

def main(): args = parse_args()

1
tokenizer = AutoTokenizer.from_pretrained(args.model_name)
2
model = AutoModelForQuestionAnswering.from_pretrained(args.model_name)
3

4
# Load dataset
5
data_files = {"train": args.train_file, "validation": args.val_file}
6
raw_datasets = load_dataset("json", data_files=data_files)
7

8
# Preprocessing function
9
def prepare_examples(examples):
10
    return tokenizer(
11
        examples["question"],
12
        examples["context"],
13
        truncation=True,
14
        padding="max_length",
15
        max_length=384
16
    )
17

18
train_dataset = raw_datasets["train"].map(prepare_examples, batched=True)
19
val_dataset = raw_datasets["validation"].map(prepare_examples, batched=True)
20

21
train_dataset.set_format("torch")
22
val_dataset.set_format("torch")
23

24
training_args = TrainingArguments(
25
    output_dir=args.output_dir,
26
    evaluation_strategy="epoch",
27
    save_strategy="epoch",
28
    num_train_epochs=args.epochs,
29
    per_device_train_batch_size=args.batch_size,
30
    per_device_eval_batch_size=args.batch_size,
31
    learning_rate=args.lr,
32
    load_best_model_at_end=True
33
)
34

35
trainer = Trainer(
36
    model=model,
37
    args=training_args,
38
    train_dataset=train_dataset,
39
    eval_dataset=val_dataset
40
)
41

42
trainer.train()
43
trainer.save_model(args.output_dir)

if name == “main”: main()#

6.3 Training and Validation
• Use relevant QA metrics like Exact Match (EM) and F1 score to track performance.
• Monitor loss curves during fine-tuning to watch for overfitting or underfitting.
• Experiment with data augmentation techniques (e.g., paraphrasing questions) in resource-scarce domains.

7. Inference Pipeline

7.1 Basic Inference
Once you have a fine-tuned QA model, you can load it for inference:

scripts/inference.py#

from transformers import AutoTokenizer, AutoModelForQuestionAnswering

MODEL_PATH = “models/checkpoints”

tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH) model = AutoModelForQuestionAnswering.from_pretrained(MODEL_PATH)

def answer_question(question, context): inputs = tokenizer(question, context, return_tensors=“pt”, truncation=True) with torch.no_grad(): outputs = model(**inputs) start_logits = outputs.start_logits end_logits = outputs.end_logits start_index = start_logits.argmax() end_index = end_logits.argmax() answer_tokens = inputs[“input_ids”][0, start_index : end_index + 1] return tokenizer.decode(answer_tokens)

if name == “main”: example_context = “Python is a high-level programming language created by Guido van Rossum.” question = “Who created Python?” print(“Answer:”, answer_question(question, example_context))#

7.2 Advanced Features
• Long Document Handling: Use chunking or a two-stage approach with a retrieval module.
• Knowledge Graph Integration: Query a knowledge base and fuse the result with an LLM for final answers.
• Reranking & Ensemble Methods: Combine multiple models or weighted scoring to boost accuracy.

8. Building a QA API

8.1 FastAPI Example
app/main.py#

from fastapi import FastAPI from pydantic import BaseModel from scripts.inference import answer_question

app = FastAPI()

class QARequest(BaseModel): question: str context: str

@app.post(“/qa”) def qa_endpoint(payload: QARequest): ans = answer_question(payload.question, payload.context) return {“answer”: ans}

@app.get(”/”) def root(): return {“message”: “Question-Answering API is running”}

8.2 Docker Deployment
Below is a sample Dockerfile:

FROM python:3.9-slim

WORKDIR /app COPY requirements.txt . RUN pip install —no-cache-dir -r requirements.txt

COPY . /app EXPOSE 8000

CMD [“uvicorn”, “app.main”, “—host”, “0.0.0.0”, “—port”, “8000”]#

Build and run:
docker build -t my_qa_app .
docker run -p 8000:8000 my_qa_app

9. Conclusions and Next Steps

Implementing a question-answering system involves multiple stages: data collection, cleaning, model fine-tuning, and an application layer for inference or deployment. By following a structured “Zero to Hero” methodology, you can incrementally build a robust QA system suited to your domain and performance needs.

Potential areas to explore:
• Multi-Hop Reasoning: Combine multiple documents or paragraphs to generate answers.
• Abstractive QA: Employ generative models to synthesize answers that may not exist verbatim in the text.
• Contextual or Conversational QA: Maintain state across multiple queries for interactive experiences.
• Domain Adaptation: Integrate knowledge graphs or specialized dictionaries for better factual accuracy.