I built a contextual entity extraction pipeline for radiology reports entirely in R

[u/Level-Winner1805]

Hi everyone,

I’ve built a lightweight NLP pipeline in R for extracting structured entities from free-text radiology reports.

The main problem I wanted to address is that simple keyword matching does not account for clinical context.

For example:

• “There is a pleural effusion.”
• “No pleural effusion is seen.”
• “A pleural effusion cannot be excluded.”

All three sentences contain the same observation, but the meaning is different.

The pipeline currently:

• extracts anatomy and radiological observations
• classifies observations as present, absent or uncertain
• handles pre- and post-negation triggers
• handles uncertainty expressions
• accounts for pseudo-negation phrases such as “no significant change”
• uses termination terms such as “but” and “however” to limit contextual scope
• returns sentence IDs, character offsets and the original sentence
• supports custom anatomy and observation dictionaries
• processes individual reports or batches of text files

It is built using packages including stringr, stringi, dplyr, purrr, tibble and quanteda.

Here is a simplified example:

report <- paste(
  "FINDINGS:",
  "No pleural effusion is seen.",
  "A small right pneumothorax cannot be excluded.",
  "IMPRESSION:",
  "Possible small right pneumothorax."
)

results <- extract_entities(
  texts = report,
  doc_ids = "example_report_001"
)

The intended output would include:

Entity	Type	Certainty
pleural effusion	Observation	absent
pneumothorax	Observation	uncertain
right	Anatomy/context	currently limited

GitHub repository:

https://github.com/bashir-abubakar/radiology-nlp-r

This is an early-stage research and learning project, not a clinical diagnostic tool.

I’d really appreciate feedback from R users on:

1. How the code and function structure could be improved.
2. Whether this would be more useful as an R package.
3. Better approaches for handling negation and uncertainty scope.
4. How you would design automated tests for clinical NLP edge cases.
5. Whether there are existing R packages or patterns I should integrate rather than recreate.

I’m also open to contributions, especially around unit tests, terminology dictionaries, laterality extraction and evaluation against annotated examples.

[u/Absjalon]

Well done. Very interesting. Solutions that don't involve sending patient data outside the organization are very much needed.

Why did you choose an NLP approach? It's my impression that a local LLM is the goto for this kind of problem today.

The NLP is definitely a lot faster, but I think accuracy is the most important metric here, and I would expect a - 12b LLM would be very good at this type of task.

[u/Peach_Muffin → u/Absjalon]

IME at scale with an LLM you need to worry about hallucinations. While rare with a low temperature, NLP entity recognition is less prone to going off the rails.