Skip to content

Add ModernBERT Masked Language Modeling notebook#3339

Open
Ilesh-Dhall wants to merge 3 commits intoopenvinotoolkit:latestfrom
Ilesh-Dhall:modernbert-masked-language-modeling
Open

Add ModernBERT Masked Language Modeling notebook#3339
Ilesh-Dhall wants to merge 3 commits intoopenvinotoolkit:latestfrom
Ilesh-Dhall:modernbert-masked-language-modeling

Conversation

@Ilesh-Dhall
Copy link
Copy Markdown

Description

This PR adds a new notebook demonstrating how to run ModernBERT (specifically answerdotai/ModernBERT-base) using OpenVINO.

ModernBERT is a modernized BERT architecture optimized for efficiency and long-context understanding (up to 8k tokens), featuring Rotary Positional Embeddings (RoPE) and Flash Attention. This notebook shows how to convert the model to OpenVINO IR and run inference for Masked Language Modeling tasks.

Use Case

The notebook covers the following steps:

  1. Model Loading: Downloads the pre-trained answerdotai/ModernBERT-base model from Hugging Face.
  2. Conversion: Converts the PyTorch model to OpenVINO IR format (.xml and .bin) using ov.convert_model.
  3. Inference: Runs static inference on a sample sentence to predict masked tokens.
  4. Interactive Demo: Launches a Gradio interface where users can input custom sentences (e.g., "The capital of France is [MASK].") and see real-time predictions powered by OpenVINO.

Validation

  • Notebook runs successfully locally.
  • Validated against CONTRIBUTING.md guidelines.
  • Code formatted with black.
  • Added test_replace metadata to Gradio cell for CI compatibility (prevents hanging on debug=True).

Gradio Screenshot

image

@review-notebook-app
Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@github-actions
Copy link
Copy Markdown
Contributor

This PR will be closed in a week because of 2 weeks of no activity.

@github-actions github-actions bot added the Stale label Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant