Prerequisites Basic computer literacy and a willingness to document experiments.
Learning objectives Explain the stage-specific model and its limits Apply the listed concepts using a controlled exercise Record results, failure cases and a reproducible evidence trail
Topics to learn Text classification, sequence labelling, retrieval and generation tasks Corpus sampling, annotation policy and class balance Train, validation and test splits without leakage Language-specific issues: Persian script, normalization and mixed-language text NLP specialization: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Tools Python Jupyter Pandas Datasheets for datasets
Hands-on exercise Create a small labelled corpus and write an annotation guide with ambiguous examples. Context for this route: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Suggested project A documented NLP problem statement, dataset card and baseline evaluation plan. The outcome must directly demonstrate NLP, not merely list tools.
Learning resources Completion criteria The task, labels, data provenance and evaluation metric are explicit and reproducible. Evidence must be reproducible and tied to NLP.
Next: Text normalization and tokenization
✓ Mark this node as complete
Prerequisites Completion of the previous stage and a working local practice environment.
Learning objectives Explain the stage-specific model and its limits Apply the listed concepts using a controlled exercise Record results, failure cases and a reproducible evidence trail
Topics to learn Unicode normalization and whitespace policy Persian/Arabic character variants and ZWNJ Sentence segmentation and tokenization Subword tokenization and vocabulary trade-offs NLP specialization: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Tools Hazm or equivalent Persian NLP tools Hugging Face Tokenizers Regex and Unicode inspectors
Hands-on exercise Build a normalization pipeline and test it against a curated set of Persian and multilingual edge cases. Context for this route: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Suggested project A tested preprocessing package with examples, unit tests and before/after samples. The outcome must directly demonstrate NLP, not merely list tools.
Learning resources Completion criteria The pipeline handles edge cases deterministically without silently changing semantic content. Evidence must be reproducible and tied to NLP.
Next: Statistical and lexical baselines
✓ Mark this node as complete
Prerequisites Completion of the previous stage and a working local practice environment.
Learning objectives Explain the stage-specific model and its limits Apply the listed concepts using a controlled exercise Record results, failure cases and a reproducible evidence trail
Topics to learn Bag-of-words and n-grams TF-IDF and sparse features Naive Bayes and linear classifiers Error analysis by label and text length NLP specialization: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Tools scikit-learn Pandas Jupyter
Hands-on exercise Train two classical baselines and compare precision, recall and macro-F1. Context for this route: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Suggested project An experiment notebook with a baseline model, confusion matrix and error taxonomy. The outcome must directly demonstrate NLP, not merely list tools.
Learning resources Completion criteria Metrics are computed on held-out data and the main error categories are explained. Evidence must be reproducible and tied to NLP.
Next: Embeddings and semantic representation
✓ Mark this node as complete
Prerequisites Completion of the previous stage and a working local practice environment.
Learning objectives Explain the stage-specific model and its limits Apply the listed concepts using a controlled exercise Record results, failure cases and a reproducible evidence trail
Topics to learn Static embeddings and their limitations Contextual embeddings Sentence embeddings and vector similarity Embedding evaluation and domain shift NLP specialization: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Tools Sentence Transformers Hugging Face FAISS for a local experiment
Hands-on exercise Build a semantic search prototype and inspect false matches and missed matches. Context for this route: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Suggested project A small semantic retrieval demo with a relevance-labelled evaluation set. The outcome must directly demonstrate NLP, not merely list tools.
Learning resources Completion criteria Retrieval quality is measured against labelled examples, not judged only by a few screenshots. Evidence must be reproducible and tied to NLP.
Next: Sequence models and Transformers
✓ Mark this node as complete
Prerequisites Completion of the previous stage and a working local practice environment.
Learning objectives Explain the stage-specific model and its limits Apply the listed concepts using a controlled exercise Record results, failure cases and a reproducible evidence trail
Topics to learn Attention and Transformer blocks Encoder, decoder and encoder-decoder models Pretraining objectives and token limits Model families and task/model fit NLP specialization: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Tools PyTorch Hugging Face Transformers Model cards
Hands-on exercise Run inference with an appropriate pretrained model and inspect tokenization and truncation effects. Context for this route: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Suggested project A reproducible model comparison for a selected language task. The outcome must directly demonstrate NLP, not merely list tools.
Learning resources Completion criteria Model choice, input constraints and observed limitations are documented. Evidence must be reproducible and tied to NLP.
Next: Task adaptation and fine-tuning
✓ Mark this node as complete
Prerequisites Completion of the previous stage and a working local practice environment.
Learning objectives Explain the stage-specific model and its limits Apply the listed concepts using a controlled exercise Record results, failure cases and a reproducible evidence trail
Topics to learn Fine-tuning vs prompting vs feature extraction Learning rate, batch size and overfitting Class imbalance and data augmentation Reproducible configuration and checkpoints NLP specialization: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Tools Transformers Trainer PyTorch Weights & Biases or MLflow
Hands-on exercise Fine-tune a small model on a bounded dataset and compare it to the statistical baseline. Context for this route: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Suggested project A versioned NLP experiment with configuration, checkpoints and evaluation report. The outcome must directly demonstrate NLP, not merely list tools.
Learning resources Completion criteria The test set remains untouched during tuning and results can be reproduced. Evidence must be reproducible and tied to NLP.
Next: Evaluation, robustness and responsible use
✓ Mark this node as complete
Prerequisites Completion of the previous stage and a working local practice environment.
Learning objectives Explain the stage-specific model and its limits Apply the listed concepts using a controlled exercise Record results, failure cases and a reproducible evidence trail
Topics to learn Macro/micro metrics and calibration Subgroup and language-variant error analysis Hallucination, bias and privacy risks where applicable Human review and confidence thresholds NLP specialization: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Tools Evaluate library Great Expectations or custom validators Error analysis notebook
Hands-on exercise Create a test suite for misspellings, dialect, mixed scripts and out-of-domain examples. Context for this route: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Suggested project A model card with measured limitations, safety checks and a review workflow. The outcome must directly demonstrate NLP, not merely list tools.
Learning resources Completion criteria Known failure cases and human escalation criteria are visible to stakeholders. Evidence must be reproducible and tied to NLP.
Next: Capstone: language product
✓ Mark this node as complete
Prerequisites Completion of the previous stage and a working local practice environment.
Learning objectives Explain the stage-specific model and its limits Apply the listed concepts using a controlled exercise Record results, failure cases and a reproducible evidence trail
Topics to learn Data ingestion and preprocessing pipeline Model serving and latency measurement Monitoring for data drift and quality regressions Documentation, privacy and handover NLP specialization: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Tools FastAPI Docker Prometheus or structured metrics Git
Hands-on exercise Package the model behind a small API and test normal, invalid and adversarial inputs. Context for this route: NLP: focus on tokenisation, linguistic preprocessing, text representation, transformer evaluation and language-specific error analysis.
Suggested project A deployable NLP service for a bounded task, with an evaluation set, tests and operations guide. The outcome must directly demonstrate NLP, not merely list tools.
Learning resources Completion criteria Another engineer can run the service, reproduce evaluation and understand data/model limitations. Evidence must be reproducible and tied to NLP.
End of roadmap: Complete the final project and build your portfolio.
✓ Mark this node as complete