DICOM vs FHIR for Healthcare AI Teams: What to Use When

If you've ever worked around healthcare AI, radiology AI, or clinical data pipelines, you've probably heard two terms again and again: DICOM and FHIR.

At first, they sound like two random healthcare acronyms people throw into meetings. And honestly, I get it. When I first started looking into healthcare data standards, DICOM and FHIR felt like the same bucket: "some format hospitals use to exchange data."

But once you actually start building healthcare AI systems, you realize they are not the same thing at all. They are more like two different languages spoken inside the same hospital.

One language is used by imaging systems like CT scanners, MRI machines, X-ray machines, PACS, and radiology viewers. The other is used by clinical systems like EHRs, patient records, labs, medications, encounters, orders, and reports.

That first language is DICOM. The second one is FHIR.

If you're building AI for healthcare, knowing when to use each one can save you from a lot of confusion, bad architecture decisions, and painful integration work later. So let's break this down in a simple way.

The easiest way to understand DICOM and FHIR

DICOM is for medical images. FHIR is for clinical healthcare data.

That's it. But let's make it practical.

Imagine a patient comes in with chest pain. The doctor orders a CT scan. The CT images, their metadata, the study details, series information, image orientation, slice thickness, and pixel spacing are all handled using DICOM.

Now imagine the same patient also has blood pressure readings, lab results, medications, allergies, diagnoses, and physician notes inside the EHR. That kind of structured clinical information is usually exchanged using FHIR.

So if you're building a model that detects lung nodules from CT scans, you'll need DICOM. If you're building one that predicts sepsis risk from vitals, labs, and medications, you'll need FHIR. And if you're building a multimodal system that uses CT images plus patient history? Then you need both.

What is DICOM?

DICOM stands for Digital Imaging and Communications in Medicine. It is the standard used for medical imaging. When a hospital stores or sends a CT, MRI, X-ray, ultrasound, or mammogram, DICOM is usually involved somewhere in that workflow.

A common mistake is to think of DICOM as just an image format. It is not. A DICOM file contains the actual image data, but it also carries a lot of metadata that makes the image clinically meaningful, the patient identifier, study date, modality (CT, MR, XR, US), body part, slice thickness, pixel spacing, image orientation, scanner information, and unique identifiers (the Study, Series, and SOP Instance UIDs).

This metadata matters a lot. A normal PNG might show a chest X-ray, but it does not preserve the acquisition details, imaging geometry, or the relationships between slices and series. DICOM was built for that world. It understands that medical images are not just pictures, they are clinical objects that need to be stored, retrieved, displayed, processed, and connected to patient care. This is exactly the kind of detail that matters when we build medical imaging AI.

A simple way to picture DICOM

Think of DICOM as the hospital's imaging filing system. It does not just say "here is an image." It says: "here is an image from this patient, this study, and this series, acquired on this machine, with this spacing, orientation, and modality."

A CT scan is rarely a single image. It can be hundreds or thousands of slices, organized into series, which together make up a study:

Patient
  └── Study (e.g. CT Chest)
        └── Series (Axial lung, Soft tissue, Coronal…)
              └── Image / Instance

For a human radiologist, this structure matters. For an AI model, it matters too. If your model is supposed to detect lung nodules, it should run on the right CT series, not on the localizer image or the wrong reconstruction. This is where DICOM becomes essential.

What is FHIR?

FHIR stands for Fast Healthcare Interoperability Resources, a standard created by HL7 for exchanging healthcare information electronically. The important word here is resources: FHIR breaks healthcare data into small, reusable building blocks, each representing a specific clinical or administrative concept.

Developers often like FHIR because it fits naturally into modern software development: FHIR APIs commonly use REST, JSON, and OAuth-style authorization, so it feels much closer to how developers already build applications than older healthcare standards do.

That does not mean integration is suddenly easy, healthcare data is still messy, and implementations still differ. But FHIR gives teams a far more modern, structured way to work with EHR data.

A simple way to picture FHIR

Think of FHIR as the hospital's clinical API layer. It helps applications ask questions like: Who is this patient? What visit is this? What are the latest lab results? What medications are they taking? What diagnoses do they have? What imaging order was placed?

FHIR is not mainly about the raw pixels inside a CT scan. It is about the clinical story around the patient, which may include imaging, but also labs, notes, medications, allergies, diagnoses, procedures, reports, and care workflows.

So, what is the real difference?

The difference is not that one is better than the other. They were designed for different jobs. DICOM is deeply specialized for medical imaging. FHIR is designed for broader healthcare data exchange.

A simple way to remember this: DICOM handles the image evidence. FHIR handles the clinical context.

Why healthcare AI teams should care

Healthcare AI teams usually care about three things: getting the right data into the model, running inference safely, and sending results back into the clinical workflow. DICOM and FHIR appear at different points in that journey.

Take a model that detects intracranial hemorrhage from head CT scans. It needs the CT image data (DICOM), but it may also need the patient's age, order details, and where the result should go, clinical data that lives in the EHR via FHIR. And once it runs, the result often has to land in two places: the radiologist's viewer as a DICOM object, and the patient's record as a FHIR DiagnosticReport, Observation, or Task. This is why real-world healthcare AI is rarely just one standard. It's usually both.

Example 1: AI for chest X-ray triage

Imagine a chest X-ray system that flags possible pneumothorax. The X-ray lives in the imaging system as DICOM, so the model retrieves it from the PACS, runs inference, and produces a finding. But what happens after it predicts "possible pneumothorax"? You may want to notify the radiologist, prioritize the study in a worklist, and attach a summary to the patient record, and that's where FHIR comes in:

DICOM image from PACS → AI model
  → DICOM result for the radiology viewer
  → FHIR result for the EHR workflow

DICOM gets the image to the model. FHIR gets the result into the workflow.

Example 2: AI for sepsis prediction

Now a non-imaging example. A sepsis model doesn't need a CT or X-ray. It needs temperature, heart rate, blood pressure, respiratory rate, white blood cell count, lactate, medications, diagnoses, and encounter information. This isn't a DICOM problem, it's FHIR territory, pulling resources like Observation, MedicationRequest, Condition, and Encounter from the EHR API and writing the risk score back as an Observation or Task. Using DICOM here wouldn't make sense, because there are no images involved.

And multimodal AI? A cancer-risk model that combines CT images with smoking history, prior diagnoses, and labs needs both: DICOM for the imaging side, FHIR for the clinical side. Each does its own job, DICOM isn't trying to become the EHR, and FHIR isn't trying to become the PACS.

Where DICOMweb fits in

You may also hear DICOMweb, basically the web-friendly way to access DICOM data. It provides RESTful services for searching studies (QIDO-RS), retrieving objects (WADO-RS), and storing them (STOW-RS). For modern, cloud-native AI teams, this is often easier than older DICOM network protocols. But it is still about imaging data, it doesn't make DICOMweb the same as FHIR.

The role of FHIR ImagingStudy

FHIR has an ImagingStudy resource, and this is where some confusion starts. People assume: "if FHIR has ImagingStudy, maybe we can use FHIR instead of DICOM for images." Not quite.

ImagingStudy can describe an imaging study and reference DICOM studies, series, and instances, but the actual image pixels are not stored inside it. Think of it like a smart pointer or index card: it tells you the patient has this study and how to retrieve it, but for the real pixels you still go back to DICOM or DICOMweb. FHIR helps you discover and connect imaging to the record; DICOM remains the source of truth for the image itself.

How AI results get stored

A practical question: once the model produces a result, do you store it in DICOM or FHIR? It depends on the result. Image-native outputs, segmentation masks, lesion contours, measurements, fit DICOM best (a DICOM SEG for masks, a DICOM SR for measurements). Outputs meant for the EHR, "possible pneumothorax detected," "follow-up recommended," "create a review task", fit FHIR better, via Observation, DiagnosticReport, or Task.

Often you store both: a DICOM output for the radiology viewer and a FHIR output for the EHR. That sounds repetitive, but the radiologist lives in the viewer while the physician lives in the EHR, and your result needs to meet both where they already work.

Common mistakes AI teams make

1. Converting DICOM to PNG too early. Convenient for a quick prototype, risky for serious work, you can lose pixel spacing, orientation, intensity information, and study/series relationships that some models depend on. Do DICOM-aware preprocessing and keep links back to the original objects.

2. Treating FHIR as an image store. FHIR is not a PACS. It can reference imaging studies, but if your model needs actual CT or MRI pixels, FHIR alone isn't enough. Use FHIR to find the patient and order; use DICOM or DICOMweb to retrieve the images.

3. Sending results to only one system. The same finding may need to appear as a lesion overlay in the viewer and a summary in the EHR. Save only a DICOM object and it may not show in the EHR; save only a FHIR observation and it may not show in the viewer.

4. Ignoring patient identity mapping. DICOM and FHIR don't always use identifiers the same way. You may need to map Patient IDs, accession numbers, Study Instance UIDs, encounter and order IDs, and MRNs carefully. A correct output attached to the wrong patient is worse than no output.

5. Forgetting de-identification. Both standards can carry protected health information, DICOM metadata (and even burned-in annotations) and FHIR demographics and notes. For training, evaluation, and sharing, de-identification has to be designed across both imaging and clinical data, not bolted on.

So, what should you use?

Use DICOM when you need to read CT/MRI/X-ray/ultrasound images, work with PACS or VNA systems, preserve imaging metadata, understand study/series/instance structure, display results in radiology viewers, or store segmentation masks and measurements.

Use FHIR when you need to read EHR data, access demographics, use labs/vitals/medications/diagnoses, understand encounters and workflow, write results back to the record, or create alerts and tasks.

Use both when you need to combine imaging and EHR data, build multimodal models, send results to both PACS and EHR, or link image findings to clinical history.

Final thoughts

DICOM and FHIR are both important, but they're not interchangeable. DICOM is what you reach for when you care about the medical image itself, scans, series, metadata, and PACS workflows. FHIR is what you reach for when you care about the clinical data and workflow around the patient, records, labs, medications, diagnoses, orders, and tasks, through modern APIs.

So the better question isn't "should we use DICOM or FHIR?" It's "which part of our AI workflow needs DICOM, and which part needs FHIR?" In most serious healthcare AI products, especially imaging ones, the answer is usually both.

Use DICOM to respect the image. Use FHIR to respect the clinical workflow. Get that boundary right early, and your AI system will be far easier to integrate, validate, deploy, and actually use in the real world. If you'd like a second pair of eyes on that boundary, it's exactly the kind of thing we de-risk in a healthcare AI proof of concept.