Generative AI and LLMs in Clinical and Laboratory Medicine

Generative AI and LLMs

in Clinical and Laboratory Medicine

• What is Generative AI and how is it transforming medicine?
  
• Why focus on Large Language Models (LLMs)?
  
• Key goals of the course:
  • 🧠 Understand the fundamentals of generative AI and LLMs
    
  • 🧪 Explore applications in clinical and lab settings
    
  • 🔄 Compare local vs commercial tools (ChatGPT, Claude, Mistral, etc.)
    
  • 🛠 Practice with LM Studio, WebLLM
    
• Course format: 2h theory + 2h practical

Welcome to the course!

We’re here to explore how Generative AI — and especially Large Language Models — are reshaping the medical and laboratory fields. This isn’t a coding class; it’s a practical, clinician-focused introduction to tools like ChatGPT, Claude, and Mistral, including those you can run locally for privacy.

Today’s structure is simple:
👉 First, we understand how these models work.
👉 Then, we get hands-on with tools like LM Studio.

By the end, you’ll be able to try out prompts for diagnostics, automate reporting, and even think critically about when not to trust an LLM.

Let’s dive in!

What is Artificial Intelligence?

(And why doctors and biologists should care)

• 🧠 Artificial Intelligence (AI) simulates human reasoning and decision-making
  
• 🤖 Two historical approaches:
  • Symbolic AI = rule-based logic (e.g., expert systems)
    
  • Connectionist AI = inspired by the brain (neural networks)
    
• 📊 Statistical Learning forms the bridge:
  • Data-driven techniques to learn patterns from data
    
  • Includes regression, classification, and clustering
    
• 🧬 Why it matters in medicine:
  • Many clinical tasks are decision problems under uncertainty
    
  • From diagnosis to triage to interpretation of lab results
    
  • AI = help, not replacement

This slide sets the stage. Before we get to transformers and LLMs, we remind everyone that AI is not magic — it’s logic and statistics at scale.

We introduce two traditional visions of AI: - Symbolic, like if–then rules (think of early expert systems). - Connectionist, where intelligence emerges from networks (like modern neural nets).

Then we highlight Statistical Learning as the bridge: the core idea that algorithms can learn from data, not just rules. That’s how we move toward modern AI in healthcare.

This matters to clinicians and lab professionals because medical decision-making often requires synthesizing complex, uncertain data — something statistical models (and later, LLMs) are surprisingly good at helping with.

Why Do We Need a New AI in Medicine?

Limitations of classical statistical models

• 📊 Classical models (regression, decision trees, etc.) work well with structured, tabular data
  
• 🧬 But clinical data is increasingly unstructured and complex:
  • Free-text reports, multi-language notes, EHRs, medical images
    
• ❌ Statistical models struggle with:
  • Language ambiguity (e.g., “it” → “the liver”?)
    
  • Missing or noisy data
    
  • Context-dependent reasoning
    
• 🧠 Modern AI (LLMs) can handle this complexity with better generalization on free-text and multimodal data

This slide sets the stage for understanding why we need modern AI. Traditional models like logistic regression or random forests are great on clean, structured datasets — for example, predicting risk scores from lab values.

But real-world clinical data is messy. A discharge summary, an MRI image, a patient’s email — none of these fit neatly into rows and columns.

Modern AI, and especially Large Language Models, can process this complexity thanks to self-supervised learning and architectures like Transformers. They can extract structure from unstructured data.

Why Do We Need a New AI in Medicine?

Limitations of classical statistical models

On the left, we see a typical example of structured data — a logistic regression curve predicting a probability (e.g., of disease) based on a lab value. This is the type of data most traditional models were built for: numerical, clean, and formatted in tables.

On the right, we see unstructured data: an EHR containing free-text (“The liver is enlarged…”) with ambiguous references, embedded lab results, and even a brain image. This is what clinicians deal with daily — complex, multimodal information not easily reducible to numbers or categories.

At the bottom center, a Transformer icon represents how modern AI models like LLMs can bridge this gap. These models are capable of processing unstructured inputs — understanding the context of text, extracting meaning, and integrating across formats — making them much more powerful in handling today’s clinical data.

What Can Traditional AI Do?

Core Tasks in Statistical Learning

• 📈 Regression: Predict a number from input data
  • Example: Predict blood glucose from age + BMI
    
• 🧪 Classification: Assign a label to input data
  • Example: “Is this biopsy malignant or benign?”
    
• 🔍 Clustering: Find hidden groups in unlabeled data
  • Example: Identify subtypes of patients with similar gene expression

These are the foundations of traditional statistical learning.

• Regression predicts continuous outcomes — useful for things like lab values or risk scores.
  
• Classification is probably the most used — determining if a patient has a disease or not based on test results.
  
• Clustering is more exploratory — no labels, just patterns. Think of how we discover new cancer subtypes from omics data.

All three are still crucial, but they struggle when the inputs are free text, images, or conversations. That’s where LLMs enter the picture.

Why Statistical Models Struggle with Language

From structured input to real-world complexity

• 🧱 Traditional models expect structured, tabular input
• 💬 Natural language is unstructured, ambiguous, context-dependent
  • Example: “It is elevated” — what is “it”?
• 🧠 Language understanding needs:
  • Context resolution (e.g., coreference)
  • Syntax and semantics
  • Long-range dependencies across sentences
• 📉 Statistical models lack memory or reasoning — they reduce text to bags of words or fixed vectors

This slide sets up the motivation for deep learning and Transformers.

Traditional models can’t “understand” language. They only count word frequency or use basic rules. But in medicine, you need to know what “it” refers to, interpret negations, or follow a patient story over multiple paragraphs.

This complexity is what made deep learning necessary — especially models like RNNs and Transformers.

What Is a Neural Network?

From neurons to layers to learning

• 🔢 Neural networks are made of units (“neurons”) connected in layers
• 🧠 Each neuron takes input → does a small math operation → passes output forward
• 📚 By adjusting weights during training, the network learns patterns
  
• 🧬 This structure lets it capture non-linear relationships (vs classical regression)

Let’s now shift from classical models to the first true building blocks of modern AI: neural networks.

A neural network is essentially a system made of very simple processing units — we call them “neurons” — that are connected in layers.

Each neuron receives some numbers — like age, glucose, or symptoms — and applies a very basic mathematical operation: it sums the inputs, applies a function (usually non-linear), and sends the result to the next layer.

This sounds trivial, but the power of a neural network comes from stacking many of these neurons into multiple layers, and from adjusting the connection weights between them during training.

The process of learning in a neural network means: - Showing it many examples (e.g. patients with known outcomes), - Comparing its predictions to the actual results (e.g. real diagnoses or scores), - And gradually changing the weights to reduce the error.

Over time, the network gets better at recognizing patterns — even complex, non-linear relationships that traditional models (like logistic regression) can’t capture.

This structure — simple units in layers — is the foundation of deep learning.

Even the most advanced models we use today, including ChatGPT, are built on this principle — just scaled up to billions of neurons and much more complex architectures.

This is the moment when AI stopped being about just rules and formulas, and started learning from data in a flexible, powerful way.

How Do Neural Networks Learn?

The role of loss and backpropagation

• 🧪 During training, the network makes a prediction
• 📉 A loss function compares the prediction to the true label
• 🔁 Backpropagation adjusts the weights to reduce error
• 🔄 This is repeated over many examples → the model learns patterns

Now that we’ve seen how a neural network is built, let’s briefly see how it learns.

Every time the network sees an input — for example, patient age, glucose, and symptoms — it makes a prediction.

That prediction is compared to the true outcome — for instance, whether the patient had a complication. The difference between the prediction and the actual result is measured using a loss function.

Then comes the key step: backpropagation.

The model calculates how much each weight contributed to the error, and uses that information to adjust the weights in the right direction — usually using a method called gradient descent.

This process is repeated thousands — even millions — of times, on many examples. And that’s how the network gradually improves: by reducing the total error across the training set.

We call this “learning”, but really, it’s error correction — just at scale and with lots of data.

Why Standard Neural Networks Struggle with Language

The need for sequential memory and attention

• 🔁 Standard networks treat inputs as fixed-length vectors
  
• 📜 But language is a sequence — word order and context matter
  
• 🧠 Early models like RNNs and LSTM were designed to process sequences
  • They use memory to retain previous words
    
• 😕 Still limited: hard to capture long-range dependencies (“it” referring 3–4 sentences back)

Neural networks are great for many tasks — but not for sequences, like language or time series.

Why? Because traditional neural nets assume the input is just a fixed-size list of numbers. But in language, word order is essential, and meaning often depends on what came before — even several sentences earlier.

This is where RNNs (Recurrent Neural Networks) were introduced: they process one word at a time and retain memory of previous inputs.

Later, LSTMs (Long Short-Term Memory) added improved memory handling.

But these models still struggle when context is long — say, in medical reports or dialogues, where “it” might refer to something 3 paragraphs earlier.

This limitation is what led to the next big leap: the Transformer.

From Deep Learning to LLMs

Why Transformers changed everything

• 🧠 Deep Learning = layered neural networks
  
• 🧱 Types of architectures:
  • MLP: good for simple inputs
    
  • CNN: excels in images and spatial data
    
  • RNN: handles sequences like speech and text
    
• ⚡ Transformer: breakthrough architecture for language
  • Uses self-attention instead of recurrence
    
  • Faster, more parallel, better at long sequences
    
• 🔁 Pretraining + Fine-tuning: strategy behind ChatGPT, Claude, Mistral

We now move from the idea of “neural networks” to the specific architectures that brought us to LLMs.

• First, the MLP multilayer perceptron — the original building block, still used in simpler tasks.
• Then, CNNs — they revolutionized image processing, and are still core to medical imaging AI.
• RNNs were our first way to work with sequences like text and speech, but they struggled with long contexts.

Then came the Transformer.
By using self-attention rather than recurrence, it scaled beautifully to large data and long texts. It became the basis for LLMs like GPT, BERT, T5, and all modern variants.

We also introduce pretraining and fine-tuning — concepts crucial to understanding how models adapt to specific tasks like medicine.

Inside the Transformer

Self-Attention and Positional Encoding

• 👁️ Self-Attention: lets the model look at all words in a sentence at once
  • Each word can “attend” to other words — capturing context and meaning
    
  • Example: in “The liver is enlarged, and it may indicate…”, what does “it” refer to?
    
• 🧭 Positional Encoding: adds order to the sequence
  • Transformers process input in parallel, so positions must be encoded explicitly
    
• 🔍 These are what let LLMs handle long-range dependencies in language
  
• 🧠 Basis for ChatGPT, BERT, and every modern LLM

This is one of the most important innovations in modern AI.

Transformers don’t rely on recurrence like RNNs or LSTMs. Instead, they look at all the words at once — and use a mechanism called self-attention to decide which ones are most relevant for each word.

For example, in the phrase “The liver is enlarged, and it may indicate…”, the model can link “it” back to “liver” using attention.

Since there’s no recurrence, Transformers can process data in parallel, making training much faster.

But because they process everything at once, they need to know where each word is in the sequence. That’s why they use positional encoding — a mathematical way to inject word order into the model.

These two ideas — self-attention and positional encoding — are what make Transformers so powerful in language tasks.

Everything that came after — BERT, GPT, ChatGPT, Claude — is built on this.

What is Self-Attention?

How Transformers “understand” language

• 👀 In self-attention, every word looks at all other words in the sentence
  • Each word decides how much to pay attention to the others
• 🧠 Example:
  • Sentence: “The liver is enlarged because it is inflamed”
  • “it” should focus attention on “liver”, not “because” or “enlarged”
• 🔄 The model builds a map of relationships between words
  • Helps resolve references, capture context, understand meaning
• ⚡ Self-attention is parallel and scales well to long texts

Self-attention is the heart of the Transformer.

Instead of reading one word after another (like an RNN), the Transformer looks at the entire sentence at once.

For each word, it asks: “Which other words are important for understanding me?”

For example, in “The liver is enlarged because it is inflamed”, the word “it” needs to connect to “liver” to make sense.

Self-attention lets the model build these connections automatically, and it can do this for all words at the same time, not sequentially.
This makes Transformers very powerful — they handle long-range dependencies and scale efficiently to big datasets.

How Does Self-Attention Work?

Three steps: Query, Key, Value

• 📚 Each word is turned into three vectors:
  • Query (Q): What am I looking for?
  • Key (K): What do I offer?
  • Value (V): What information do I carry?
• 🔍 Attention Score between two words:
  • Multiply Query of one word × Key of another
• 🧮 Then, the scores are normalized and used to mix the Values
• 🎯 This gives each word a new, context-aware representation

Now we dive a little deeper into how self-attention is actually calculated.

Each word is first transformed into three vectors: - A Query — asking “what am I looking for?” - A Key — saying “what do I have to offer?” - A Value — carrying the actual information content.

To find out how much attention one word should pay to another, we multiply the Query of the first word by the Key of the second word.

This gives a score: a measure of how important that second word is to the first.

All the scores are normalized (typically using a softmax function) so they add up to 1, and then used to combine the Values from all words.

The result is a new vector for each word, but now enriched with information from the whole sentence.

That’s how the model captures complex relationships — efficiently and in parallel.

Medical Examples of Self-Attention

How LLMs resolve ambiguity in clinical text

• 📋 Clinical Note Example: > “Patient presented with RUQ pain. Ultrasound showed a hypoechoic lesion. > CT confirmed it was a hemangioma. It measured 2.3 cm.”

• 🧠 Ambiguity Challenge: Which “it” refers to what?

  • First “it” = the lesion (connecting back to previous sentence)
  • Second “it” = the hemangioma (immediate previous reference)

• 👁️ Self-attention allows the model to create these connections automatically

This slide demonstrates how self-attention works in a real clinical context. The ambiguous pronoun “it” appears twice, and the model must determine what each instance refers to.

In traditional NLP, this would be extremely difficult, but attention mechanisms allow the model to assign different weights to different words in the context, effectively “looking back” to find the most relevant noun.

This is particularly important in clinical documentation, where referential ambiguity is common and resolution errors could lead to serious misunderstandings.

What is Positional Encoding?

How Transformers know the order of words

• 🧠 Transformers see all words together — but they need to know their order
• 🧭 Positional Encoding adds information about position to each word
• 🔢 Two ways to encode position:
  • Add fixed patterns (e.g., sine and cosine functions)
  • Or learn position embeddings during training
• 🧩 Without positional encoding:
  • The model would treat sentences like unordered bags of words!

Transformers are special because they process all words in parallel — no word comes “before” another by default.

But in language, order matters.

For example, “The patient was treated successfully” is very different from “The patient successfully treated.”

To fix this, we use Positional Encoding.

Each word gets a small “position tag” — either a fixed mathematical pattern (like sine/cosine curves) or a learned embedding.

Without this, a Transformer would lose the sense of sequence — treating the words like an unordered list.

Thanks to positional encoding, Transformers can understand complex sentence structures while processing everything in parallel.

What are Word Embeddings?

Turning words into numbers

• 🔢 Computers need numbers, not text
• 🧠 Embedding = representing a word as a vector of numbers
• 📚 Similar words → similar vectors
  • “liver” close to “kidney”, far from “car”
• ➡️ Embeddings capture meaning from large corpora

Computers can’t directly understand words — they process numbers.

Embeddings solve this by transforming each word into a vector of numbers — a list like [0.1, 0.3, -0.2, …].

The key idea is similar words have similar vectors: - “Liver” and “Kidney” are close. - “Liver” and “Car” are far.

This mapping is learned automatically by models like Word2Vec, GloVe, and also during the pretraining of Transformers.

Thus, embeddings capture semantic meaning from how words appear together in large text collections.

Embeddings in Transformers

First step before Attention

• 🏗️ Each input word is mapped to its embedding vector
• ➡️ Then positional encoding is added
• ⚡ The combined vector enters the Transformer layers
• 🎯 Embeddings are fine-tuned during training

In a Transformer, the very first operation is to turn each input word into an embedding vector.

Then, positional encoding is added to that embedding to insert order information.

This combined vector — meaning + position — is the true input to the attention layers.

Importantly, embeddings are not fixed: during training (fine-tuning), the model can adjust them for the specific task — like diagnosis, report generation, or other clinical applications.

Thus, embeddings form the foundation upon which all higher understanding is built.

What is Pretraining?

Teaching a model general language skills

• 📚 Pretraining = training a model on huge text datasets
• 🧠 Goal: learn grammar, facts, reasoning patterns
• 🔄 No specific task — model predicts missing words or next words
• 🌍 Data sources: books, websites, medical papers, conversations
• 🎯 Result: a general-purpose model ready for adaptation

Pretraining is like giving a model a huge education.

We expose it to millions or billions of sentences — covering topics from daily conversation to medicine, law, science, and more.

The model’s goal isn’t to solve a particular task but to learn: - How language works (syntax, semantics) - Basic world knowledge - Common sense reasoning

Typical tasks during pretraining: - Predict the next word in a sentence (autoregressive) - Fill in missing words (masked language modeling)

At the end of pretraining, we get a generalist model — good at understanding and generating language but not specialized yet for a specific job (like clinical diagnosis or lab report writing).

What is Fine-tuning?

Specializing the model for a specific task

• 🔧 Fine-tuning = additional training on specific datasets
• 🧪 Goal: adapt the model to medical, clinical, or lab tasks
• 🏥 Examples:
  • Predict disease from symptoms
  • Summarize lab results
  • Generate medical reports
• 🎯 Result: a specialized model focused on a domain

After pretraining, the model is a “jack of all trades” — good, but not optimized for specific jobs.

Fine-tuning is like sending the model to medical school!

We train it again, but this time: - On specialized datasets (e.g., clinical notes, lab reports, patient histories) - For specific goals (diagnosis, triage, report writing)

The model learns to focus on what matters in that domain: - Medical terminology - Typical reasoning patterns - Critical facts

After fine-tuning, the model becomes much better at clinical and laboratory tasks, but it might also “forget” some general knowledge — it’s more domain-specialized.

Interpretability in LLMs

Why understanding model behavior matters

• 🔍 Interpretability = understanding how and why a model gives a response
• 🧠 Important in clinical and laboratory settings:
  • Explain predictions and recommendations
  • Build trust with users and patients
• ⚙️ Emerging techniques:
  • Attention visualization
  • Feature attribution (e.g., SHAP, LIME)
  • Chain-of-thought prompting
• 🚨 Challenge: LLMs are complex and not fully transparent

Interpretability means being able to understand and explain a model’s outputs.

In clinical contexts, this is crucial: - Doctors need to know why a model suggests a diagnosis or treatment. - Patients need to trust AI-generated advice.

Some tools can help: - Attention maps: visualize which parts of the input the model focused on. - Feature attribution methods: like SHAP and LIME, which estimate what inputs influenced the prediction. - Chain-of-thought prompting: making the model “explain its reasoning” step by step.

However, current LLMs are so large and intricate that full transparency is still very hard. Interpretability remains an open research field!

Limitations of LLMs

Hallucinations and Algorithmic Bias

• 🎭 Hallucinations: model invents plausible but false information
  • Danger: confident but wrong answers
• ⚖️ Algorithmic Bias: model reproduces biases from training data
  • Risk: unfair outcomes for certain groups
• 🚑 In clinical use:
  • Always require human validation
  • Prefer domain-specific fine-tuning

Two major risks with LLMs are:

• Hallucinations: when the model “makes up” facts that sound believable but are incorrect.
  • In medicine, this can be dangerous if wrong information is used for decision-making.
• Algorithmic Bias: models can unintentionally reproduce societal or dataset biases (e.g., racial, gender, socioeconomic).

In clinical and laboratory fields: - LLM outputs must always be reviewed by professionals. - Training or fine-tuning on high-quality, curated medical data can reduce but not eliminate these problems.

AI should support, not replace, critical thinking!

Medical Examples of Hallucinations

When LLMs fabricate clinical information

• 🩺 Prompt: “What are the normal ranges for liver function tests?”

• ✅ Accurate responses:

  • “ALT normal range: 7-56 U/L”
  • “AST normal range: 8-48 U/L”

• ❌ Hallucinated responses:

  • “GGT normal range: 15-30 U/L” (actual: 9-48 U/L)
  • “Albumin: 3.5-6.0 g/dL” (actual: 3.5-5.0 g/dL)

• 🚨 Clinical dangers:
  • False confidence in inaccurate values
  • Reference ranges vary by lab/population
  • Subtle errors harder to detect than obvious ones

This slide illustrates how hallucinations manifest in clinical applications. The model provides a mix of accurate and inaccurate information, but presents all with the same level of confidence.

What makes this particularly dangerous is that the hallucinated values aren’t wildly incorrect - they’re plausible enough that someone might not catch them without checking. This is the “plausible but wrong” problem we discussed earlier.

In clinical settings, these subtle hallucinations can be more dangerous than obvious errors because they’re less likely to trigger skepticism or verification.

LLMs in Clinical and Laboratory Medicine

Pros and Cons

✅ Advantages	⚠️ Limitations
Fast information retrieval	Hallucinations: plausible but wrong answers
Assist in decision support	Algorithmic bias from training data
Summarize complex medical texts	Lack of full interpretability
Help generate reports and documentation	Risk of overconfidence in outputs
Available 24/7, scalable support	Dependence on human validation for safety

Here we summarize the main advantages and limitations of using LLMs in clinical and laboratory settings.

✅ Advantages: - Speed up information access and reporting. - Support clinicians with suggestions and summaries. - Operate continuously, without fatigue.

⚠️ Limitations: - Risk of hallucinating wrong but convincing answers. - Potential biases inherited from imperfect data. - Opaque reasoning: we often can’t fully understand why a model gives a certain answer.

Thus, LLMs are powerful assistants, but require expert supervision in all critical decisions.

LLMs: Pleasing You, Not Seeking Truth

Why plausible ≠ correct

• 🎭 LLMs are trained to sound convincing, not to tell the truth
• 🤝 Goal: produce answers that seem helpful, coherent, pleasant
• ❌ Risk: if unsure, the model guesses plausible but wrong facts
• 🚨 Danger in clinical settings: wrong information can look very credible
• 🧠 Always require critical review and human validation

Large Language Models are not designed to find the truth — they are designed to predict the most likely next words.

Their goal is to please the user, giving answers that sound fluent, confident, and helpful.

This means: - When they “know”, they are often brilliant. - When they “don’t know”, they still produce an answer, even if it’s wrong. - They don’t say “I don’t know” unless explicitly programmed or instructed to do so.

In clinical contexts, this tendency is very dangerous: - Errors can be hidden behind perfectly structured language. - Clinicians must always critically evaluate LLM outputs. - No LLM output should ever replace professional judgment.

Teach users: Plausibility is not correctness!

⚠️ Warning: Plausibility is NOT Truth

Warning

• A fluent, confident answer can still be wrong.
• LLMs are rewarded for sounding helpful, not for being accurate.
• In clinical and laboratory applications: always validate before trusting.

Key Model Parameters in LLMs

How to control AI behavior

• 🌡️ Temperature: randomness level (higher = more creative, lower = more focused)
• 🎯 Top-p (nucleus sampling): limit choices to most probable words
• ✂️ Max tokens: maximum length of the output
• 🔁 Frequency penalty: discourage repeating the same words
• 🧠 Fine-tuning these helps adapt the model to clinical needs

Let’s now take a quick but important look at key parameters that control how an LLM responds.

The first is temperature.
This defines the randomness of the model’s outputs:
- A higher temperature — for example, 0.7 or 1.0 — makes the model more creative, but also more unpredictable.
- A lower temperature — like 0.2 or 0.1 — makes it more focused and repetitive.
In clinical work, we usually prefer lower temperatures to get consistent, safe answers.

Second is top-p, also known as nucleus sampling.
It restricts the model to choose from the top most probable words only, instead of exploring the whole vocabulary.
Smaller top-p values make the model more conservative.

Third is max tokens — the maximum number of words or characters the model is allowed to generate.
This prevents it from producing endless outputs or going off-track.

Finally, there’s the frequency penalty, which tells the model:
“Don’t repeat yourself too much.”
Useful when you want summaries that are concise and not redundant.

Tuning these parameters properly makes a huge difference — especially in sensitive fields like medicine, where we want reliable, predictable outputs.

Temperature Settings for Clinical Use

Finding the right balance between creativity and accuracy

• 🌡️ Temperature scale:
  • 0.0-0.3: Most deterministic, consistent
  • 0.4-0.7: Balanced creativity
  • 0.8-1.0: Maximum creativity, unpredictability
• 🩺 Clinical recommendations:
  • Patient documentation: 0.1-0.2
  • Differential diagnosis: 0.3-0.5
  • Patient education materials: 0.4-0.6
  • Research brainstorming: 0.7-0.8

• ⚠️ Example impact:

  Temperature 0.1: > “Elevated liver enzymes may indicate hepatocellular injury.”

  Temperature 0.7: > “Elevated liver enzymes could suggest hepatocellular damage, biliary obstruction, medication effects, or various systemic conditions.”

Let’s now talk about a crucial parameter that’s often overlooked: temperature. This parameter essentially controls how “creative” or unpredictable the model can be in its responses.

A low temperature, between 0.0 and 0.3, produces very consistent and conservative responses. The model will almost always respond the same way to the same question. A medium temperature, between 0.4 and 0.7, allows more variability. With high temperatures, between 0.8 and 1.0, we get very creative but potentially unpredictable responses.

In clinical settings, these differences are fundamental. For patient documentation, we want maximum precision and consistency, so I recommend very low temperatures, between 0.1 and 0.2. The risk of errors or hallucinations must be minimized.

For generating differential diagnoses, a slightly higher temperature (0.3-0.5) is appropriate, because we want the model to consider different possibilities without becoming too speculative.

For patient educational materials, we can go up to 0.4-0.6, allowing more natural and varied language. And for research brainstorming, where creativity is valuable, temperatures between 0.7 and 0.8 can generate innovative ideas.

Look at the examples on the right: at temperature 0.1, the model provides a concise and cautious response about liver enzymes. At 0.7, it explores a wider range of possibilities instead. Both are correct, but they serve different purposes.

Remember: in medicine, there is no “right” temperature - it depends on your specific goal and acceptable level of risk.

What Are Tokens?

The building blocks of language models

• 🧩 Tokens = small pieces of text (words, parts of words, symbols)
• 📏 Models process text token by token, not character by character
• 🧮 1 word ≈ 1–3 tokens (depending on language and complexity)
• ✂️ Max tokens limits the total input + output length
• ⚡ Costs and speed often depend on the number of tokens used

Now let’s quickly explain what a token actually is, because it’s fundamental to understanding how LLMs work.

A token is a small chunk of text.
It could be a word, part of a word, a symbol, or even a piece of punctuation.

For example: - The word “hospital” is usually one token.
- The word “discharge-summary” might be split into two or three tokens, depending on the model’s tokenizer.

Models don’t read full sentences at once — they process text token by token, step by step.

On average, one English word equals about 1 to 1.5 tokens — but in more complex languages or technical texts, it might go up to 2 or 3 tokens per word.

The number of tokens is extremely important because: - It limits how much text you can send and receive in one call. - It affects costs if you’re using a paid API. - It impacts speed: more tokens mean slower responses.

In practical terms:
Always check the token limits of your model — especially when summarizing large clinical documents or doing batch processing.

Example: How Text Becomes Tokens

Real clinical sentence broken into tokens

• 📄 Sentence: > “Patient discharged in stable condition.”

• 🧩 Tokenization:

  • “Patient”
  • ” discharged”
  • ” in”
  • ” stable”
  • ” condition”
  • “.”

• 🔢 Total: 6 tokens

✅ Even short sentences can use multiple tokens!

Let’s see a real example of how tokenization works.

Take this simple clinical sentence:
“Patient discharged in stable condition.”

When processed by a language model, it doesn’t see it as one big chunk.
It splits it into smaller parts, called tokens.

The tokens would be:
- “Patient”
- ” discharged”
- ” in”
- ” stable”
- ” condition”
- “.”

Notice how even small spaces and prefixes (” discharged” with a space) can become separate tokens.

In total, this tiny sentence uses 6 tokens.

This is why, when working with long clinical texts like discharge letters or radiology reports, token limits can become a real constraint — even if the text doesn’t look very long to a human reader!

How LLMs Process Images

Turning pictures into language

• 📸 Images are converted into numerical features (arrays of numbers)
• 🔎 Vision encoder extracts key elements: shapes, colors, objects, text
• 🧠 Features are interpreted by the language model
• 🖋️ Model generates descriptions, answers, or captions based on visual inputs
• ⚡ Clinical use: analyzing X-rays, MRIs, pathology slides, diagrams

Let’s quickly see how modern multimodal LLMs can process images and similarities to words processing.

First, when you send an image to the model, it doesn’t see a picture like we do.
Instead, the image is converted into a numerical array — a long list of numbers representing colors, shapes, and pixels.

A special component called a vision encoder analyzes the image.
It extracts key visual features: - Objects - Text - Positions - General patterns

These extracted features are then fed into the language model, which interprets them just like a prompt in text form.

Finally, the model can describe the image, answer questions about it, or reason based on what it “saw”.

In clinical practice, this approach opens possibilities like: - Automatically describing X-rays - Summarizing findings from MRIs - Interpreting pathology slides - Reading hand-written notes in scanned PDFs

The bridge between vision and language is what makes multimodal models so powerful.

Multimodal Applications in Medicine

Beyond text: LLMs with vision capabilities

• 🔬 Clinical applications:
  • Describing radiological images
  • Interpreting ECG patterns
  • Analyzing microscopy slides
  • Reading handwritten medical notes
• ⚡ Workflow examples:
  • Upload image + add clinical question
  • Model interprets visual + text context
  • Response incorporates both modalities

• 🔍 Example prompt: > “This is a chest X-ray from a 65-year-old patient with shortness of breath. Describe what you see and any potential abnormalities.”

• ⚠️ Limitations:

  • Not FDA-approved for diagnosis
  • Variable performance across image types
  • Requires clinical verification

Let’s enter the exciting territory of multimodal models - LLMs that can process both text and images. This is a true revolution for medicine, where visual information is often crucial.

Models like GPT-4V, Claude 3, and Gemini can literally “see” medical images and discuss them in clinical context. The applications are enormous: they can describe radiological images, interpret ECG patterns, analyze microscopy slides, and even read handwritten medical notes that traditional OCR systems would struggle to decipher.

The workflow is intuitive: you upload an image, add a clinical question, and the model integrates both modalities in its response. For example, you can upload a chest X-ray and ask “What do you see in this X-ray of a 65-year-old patient with shortness of breath?” The model will describe the visible findings considering the clinical context provided.

This is particularly powerful because it combines visual analysis with medical knowledge. It doesn’t just describe the image, but interprets it in light of the clinical history.

Of course, there are important limitations. These systems are not FDA-approved for diagnosis - they are assistance tools, not substitutes for clinical judgment. Performance varies significantly between different imaging modalities and image types. And every output requires clinical verification.

But the potential is undeniable. We’re seeing radiologists use these tools to generate preliminary report drafts, pathologists using them to compare samples with similar cases, and cardiologists accelerating routine ECG interpretation.

The key is to use them as intelligent assistants, not as substitutes for expert judgment.

General vs Medical-Specific Language Models

Choosing the right tool for clinical applications

• 🌍 General LLMs
  Trained on vast internet data ➔ Versatile but shallow in medicine.
  Examples: ChatGPT, Claude, Mistral.

---

• 🩺 Medical-specific LLMs
  Trained on clinical records, guidelines, scientific papers ➔ Accurate but less flexible.
  Examples: PathChat, BrainGPT, LiVersa.

---

Note

⚖️ Key trade-off:
Broad skills (general models) vs Deep expertise (medical models)

Now let’s explore a fundamental distinction if you want to use AI properly in clinical or laboratory work.

We have two major categories of language models:

On one side, there are the general-purpose models, like ChatGPT, Claude, and Mistral.
These models are trained on massive amounts of general internet data.
This makes them extremely versatile — they can reason, summarize, improvise across many domains.
But here’s the catch: they don’t really understand medicine.
They can sound very confident even when they use the wrong terminology or miss important clinical nuances.

On the other side, we have the medical-specific models, such as PathChat, BrainGPT, and LiVersa.
These models are trained exclusively on clinical datasets: patient records, guidelines, and scientific papers.
The result?
- Much better medical terminology,
- Stronger contextual accuracy,
- Higher safety when handling clinical information.
However, they tend to be less flexible outside their domain, and many are closed or institution-specific.

Bottom line:
When choosing an LLM for clinical work, you have to balance two needs: - Generality versus specialization, - Availability versus reliability.

There’s no universal answer — it depends entirely on the specific task and the level of risk involved!

Specialty-Specific Applications

LLM use cases across medical disciplines

• 🫀 Cardiology:
  • ECG interpretation assistance
  • Heart failure management protocols
• 🧠 Neurology:
  • Cognitive assessment documentation
  • Seizure pattern description
• 🔬 Pathology:
  • Standardized specimen reporting
  • Literature search for rare findings
• 🩸 Laboratory Medicine:
  • Test interpretation guidance
  • Protocol documentation and standardization
  • Complex test sequence planning

I know we have specialists from different medical disciplines in this room, so I want to show how LLMs can be specifically applied in your areas of expertise.

For cardiologists, LLMs can assist in ECG interpretation, highlighting potentially relevant patterns. They’re particularly useful for standardizing heart failure management protocols, personalizing them for specific patient subpopulations.

In neurology, these models excel in documenting cognitive assessments, where they can structure complex observations into standardized formats. They can also assist in detailed description of seizure patterns, improving documentation consistency across different operators.

Pathologists are already using LLMs to standardize specimen reports, ensuring that all necessary elements are included and consistently formatted. A particularly powerful application is rapid literature search when encountering rare or unusual findings - a model can analyze thousands of articles in seconds and synthesize relevant information.

In laboratory medicine, LLMs are revolutionizing the creation of interpretive guides for complex tests, standardizing protocol documentation, and planning complex test sequences. They can help determine which tests should follow initial abnormal results, based on updated guidelines.

These applications are just the tip of the iceberg. The key idea is that LLMs are not generic tools - they can be adapted to the specific needs of your discipline. I invite you to think about documentary or decision-making processes in your daily practice that might benefit from intelligent assistance.

Chat Mode vs API Mode

Two ways to interact with LLMs

• 💬 Chat Mode:
  ➔ Interactive, no coding required.
  ➔ Best for brainstorming, exploration.
  ➔ ❗ Less control and reproducibility.

---

• 🔗 API Mode:
  ➔ Structured programmatic queries.
  ➔ Best for automation, scalability.
  ➔ ✅ Full control over outputs.

Note

⚡ Key Tip:
Use Chat Mode to explore.
Use API Mode to automate.

Here we arrive at a very practical distinction you must understand if you want to actually work with LLMs.

There are two main ways to interact with these models:

First, the Chat Mode.
This is the most familiar — like using ChatGPT in your browser.
It’s interactive: you send a prompt, get an answer, and you can keep refining your request step-by-step.
It’s perfect for exploring, brainstorming, and rapid prototyping clinical ideas — for example, helping draft a lab report or summarizing guidelines.
The beauty is: no coding skills are needed.
But — and this is important — you have less control.
You can’t easily automate or repeat tasks exactly the same way every time.

Then we have the API Mode.
Here, instead of chatting, you send requests programmatically — via code or simple scripts.
You can automate workflows: batch-analyze clinical reports, generate hundreds of summaries, integrate LLMs into hospital systems.
It requires minimal coding: basic HTTP requests, easy to learn even for non-developers.
The huge advantage is precision and scalability — you define exactly what you want, every time.

Bottom line:
- Use Chat Mode to explore and think.
- Use API Mode to build and scale your solutions.

Both are powerful, but they serve very different needs.

How Chat Mode Actually Works

The model re-reads everything every time

• 📚 Every new message = model re-reads all previous conversation
• 🔄 Chat history + new user message are sent again at every turn
• 📈 Cost and response time grow with chat length
• 🧠 Model does not have memory between sessions: only current context

Now let’s clarify how chat mode actually works inside an LLM.

Every time you send a new message, the model doesn’t just “continue” from where it left off.
Instead, it re-reads the entire conversation history, plus your new message.

At every turn, the model receives: - All previous messages - Plus the new user input

It processes the entire combined text from scratch — as if it were seeing it for the first time.

This means: - Cost increases because more tokens are processed. - Response time increases because more text has to be analyzed. - Risk of exceeding the context window increases.

And very importantly:
The model does not have persistent memory between chats or sessions.
Everything it knows is only what you send right now in the current context.

Context Window: How Much an LLM Can Remember

Why token limits matter for conversations

• 🧠 Context window = maximum number of tokens model can process at once
• 📏 Includes both your prompt and the model’s reply
• 🚫 If the conversation exceeds the limit, old tokens are dropped (“forgetting”)
• 📉 Long chats may lose important earlier information
• 🔍 Practical tip: keep prompts concise, summarize when needed

Now let’s explain a very important but often overlooked concept:
the context window of a language model.

The context window defines the maximum number of tokens the model can handle at once.
This includes both: - The text you write (the prompt) - And the text the model generates (the response)

For example: - GPT-3.5 has a limit of about 4,096 tokens. - GPT-4 can handle up to 8,192 or even 32,768 tokens in some versions. - Local models like Mistral typically handle 4,000 to 8,000 tokens.

But what happens if your conversation becomes longer than this limit?

The model starts to forget.
Older tokens are dropped automatically.
That means: if your chat is too long, the model might forget important details from the beginning.

This is especially risky in clinical or scientific applications where every detail matters.

Context Windows and How Much They Cover

Memory limits of major LLMs

Model	Context Window	Approx. Pages
🤖 GPT-3.5	~4,096 tokens	~10 pages
🧠 GPT-4 (standard)	~8,192 tokens	~20 pages
🧠 GPT-4 (extended)	~32,768 tokens	~80 pages
🤯 Claude 3	~200,000 tokens	~500 pages
🌟 Gemini 2.5 Pro	~2,000,000 tokens	~5,000 pages
🧩 Mistral 7B	~8,192 tokens	~20 pages
🦙 Llama 2 13B	~4,096 tokens	~10 pages

Let’s now update our view of context window sizes — and make it practical by thinking in terms of book pages.

GPT-3.5, the classic model behind ChatGPT, handles about 4,000 tokens — roughly 10 pages of text.

GPT-4, in its standard form, covers about 8,000 tokens, or about 20 pages.
Its extended version, available in premium services, can process about 80 pages — really powerful for longer reasoning tasks.

Claude 3, by Anthropic, is remarkable: it can handle around 200,000 tokens, roughly 500 pages — that’s almost a full textbook.

Gemini 2.5 Pro, Google’s latest model, is even more extreme:
It claims to process 2 million tokens — about 5,000 pages.
At that scale, you could feed entire research libraries into the model.

Mistral 7B, a popular open-source model, handles around 8,000 tokens, comparable to GPT-4 standard.

Llama 2 13B — another open model — handles around 4,000 tokens.

The big takeaway is simple:
Modern models vary enormously in how much information they can “see” at once.
Choosing the right model and designing prompts that fit the context window are key to success, especially in clinical, legal, or academic settings.

What Happens When You Exceed the Context Window?

How LLMs handle too much information

• ⏳ When token limit is exceeded, oldest tokens are dropped
• 🧠 Model “forgets” early parts of the conversation
• 🚑 Critical instructions may be lost
• 📉 Performance and consistency degrade
• 🧹 Practical tip: summarize or restate key points periodically

Now let’s discuss what happens when we exceed the model’s context window.

When the number of tokens — counting both your prompts and the model’s replies — goes beyond the allowed limit, the model automatically drops the oldest tokens.

It keeps only the most recent tokens that fit within the maximum window.

This means the model starts to “forget” the beginning of your conversation:
- Important instructions - Critical details - Definitions you gave early on

Practically, this can severely affect performance — especially in clinical, legal, or technical discussions where early context matters.

Given how chat mode works — re-sending the entire history every time — there are some very practical consequences you need to manage carefully.

By managing the context well, you maintain accuracy, control cost, and avoid errors caused by missing or forgotten information.

Context Window Limitations in Clinical Practice

What happens when medical documents exceed token limits

• 📄 Typical discharge summary: 500-1000 words = ~750-1500 tokens

• ⚠️ Truncation risks:

  • Previous medical history may be cut off
  • Medication information at document end might be lost
  • Follow-up instructions could be missing

• 💡 Clinical example: Patient summary with medication list at the end

  • With 4K tokens: Complete information processed
  • With 2K tokens: Critical anticoagulation instructions lost

This slide illustrates the practical implications of context window limitations when working with clinical documents.

Medical records tend to be lengthy, and important information is often distributed throughout the document. When a model can only “see” part of the document due to token limitations, critical information may be missed.

In the example shown, medication instructions placed at the end of a discharge summary could be lost entirely if the context window is too small, potentially leading to treatment errors.

This directly impacts how we should structure prompts for clinical applications, and highlights why newer models with larger context windows (like Claude and GPT-4) represent significant improvements for healthcare applications.

Protecting Context in Long Conversations

Techniques to avoid critical information loss

• 🛡️ Anchor Instructions:
  • Repeat critical rules or instructions every few prompts
• 📝 Summary Injection:
  • Summarize key points and re-feed them during the chat
• 📚 Structured Prompting:
  • Organize inputs clearly: diagnosis, treatments, follow-up
• 🚦 Short Sessions:
  • Restart new chats after reaching 70–80% of the token limit

Let’s now talk about practical techniques to protect important context in long conversations with LLMs.

The first technique is called Anchor Instructions:
You repeat key instructions — like “Summarize only clinical information” — every few prompts.
This keeps the model focused, even if early tokens are dropped.

The second method is Summary Injection:
Periodically, you summarize the conversation so far and re-inject the summary into the dialogue.
It acts like a memory refresh for the model.

Third, use Structured Prompting:
When giving clinical or technical information, organize it explicitly:
- “Diagnosis: …”
- “Treatments: …”
- “Follow-up instructions: …”
This helps the model parse and maintain the logic of your input, even when memory shrinks.

Finally, practice Short Sessions:
When you see that a conversation is getting long — and you estimate you’re at 70% or 80% of the model’s token limit — it’s safer to start a new session, copying over only the critical information.

These techniques can dramatically improve consistency and safety when using LLMs for complex tasks like clinical summarization or diagnostics.

What are APIs?

Connecting to LLMs like professionals

• 🔗 API = Application Programming Interface
• 🛠️ A way to send questions and receive answers programmatically
• 📬 Works like “sending a message” to the model and getting a reply
• ⚡ Allows automation, scaling, and integration into clinical systems
• 🧠 No need to “chat” manually — workflows happen automatically

Let’s take a quick moment to define what an API actually is.

API stands for Application Programming Interface.
It’s a way to send a request to a model — like a question or a prompt — and receive a response automatically.

Instead of chatting manually, an API lets you connect your software directly to the model.
You send a structured request, usually through a simple script, and you get a structured answer back.

Think of it like sending a message to a robot — and getting a smart reply instantly, without having to open a chat window.

APIs are what enable automation:
- You can process hundreds of reports overnight.
- You can integrate the model directly into hospital systems, electronic health records, lab information systems.

And all of this happens without manual interaction: just programs talking to each other.

In clinical practice, APIs are powerful because they make LLMs truly scalable and operational — not just a tool for brainstorming.

Quantitative Performance Evaluation

Measuring LLM effectiveness in clinical tasks

• 📊 Key metrics:
  • Accuracy: Correctness of medical information
  • Consistency: Reliable responses to similar queries
  • Hallucination rate: Frequency of fabricated content
  • Clinical relevance: Applicability to patient care
• 🔍 Evaluation methods:
  • Expert review panels
  • Comparison to gold standards
  • Inter-model consistency checks
  • Structured clinical scenarios

• 📈 Sample results comparison:

Model	Accuracy	Hallucination Rate
GPT-4	89%	4.5%
Claude 3	91%	3.2%
Mistral	85%	6.7%
Med-PaLM	93%	2.8%

As healthcare professionals, we’re used to rigorously evaluating new tools and therapies before adoption. LLMs should be no different. Let’s analyze how to quantitatively measure their performance in clinical settings.

Key metrics we need to consider include accuracy - how correct is the medical information provided; consistency - how reliable are responses to similar queries; hallucination rate - how often the model invents information; and clinical relevance - how applicable are the responses to patient care.

To evaluate these metrics, we can use several methods. Expert review panels, where specialists evaluate responses, are the gold standard but time-consuming. Comparison with recognized standards, like guidelines or medical texts, is more efficient. Inter-model consistency checks can reveal informational discrepancies. And structured clinical scenarios allow standardized testing on specific cases.

The table shows some comparative results from a recent study of ours. Notice how models specialized for medicine, like Med-PaLM, tend to outperform generic models. The hallucination rate is particularly important - it represents how often the model generates false but plausible information.

These data suggest we’re reaching interesting performance levels, but we’re still far from 100% accuracy. That’s why human verification remains essential, especially for information critical to patient care.

When implementing these tools in your contexts, I recommend conducting similar evaluations, specific to your use cases, to understand the strengths and limitations of the models you choose.

Practical Example: Chat Mode

Clinical Prompt for Exploration

• 🧪 Scenario: drafting a discharge summary
• 💬 Prompt:
  “Summarize the patient’s hospital stay focusing on diagnosis, treatment, and follow-up instructions.”
• ⚡ Goal: fast text generation for clinician review
• ⚠️ Reminder: always validate for accuracy and clinical relevance

In this example, we are using Chat Mode to assist with a typical clinical task: writing a discharge summary.

The clinician provides a free-text prompt, asking the model to summarize the patient’s hospital stay — focusing specifically on the diagnosis, the treatments performed, and what instructions were given for follow-up.

This kind of task is perfect for Chat Mode because it allows for fast text generation, where the human clinician can immediately review, edit, and approve the draft.

However — and this is critical — the model may sound extremely confident even if it misrepresents details.
So even when Chat Mode saves time, clinical validation is always necessary before using the output officially.

The LLM should be seen as a writing assistant, not a decision-maker.

Practical Example: API Mode

Automating Clinical Workflows

• 🔗 Scenario: batch processing of lab reports
• 🛠️ API Call:
  Send 100 lab report texts via API, receive 100 clinical summaries
• ✅ Advantage: automation, reproducibility, efficiency
• ⚠️ Reminder: monitor outputs for consistency and medical soundness

Here we move into API Mode — where instead of working interactively with one case at a time, we automate entire workflows.

Imagine a hospital system where hundreds of lab reports are generated every day.
With an API, we can batch-send these reports to the LLM, and automatically receive structured clinical summaries ready for validation.

The main advantages here are: - Speed: dozens or hundreds of cases processed quickly. - Reproducibility: the same instructions are applied consistently.

But again: even at scale, clinical supervision is critical.
Errors, misinterpretations, or hallucinations can occur — and in medicine, one mistake can have serious consequences.

Automation amplifies efficiency — but also amplifies errors if left unchecked.

Commercial vs Open-source LLMs

Comparing two worlds in clinical AI

• 🏢 Commercial models (e.g., ChatGPT, Claude, Gemini)
  • Closed source, proprietary
  • Strong performance, constant updates
  • Privacy concerns, limited control
• 🧪 Open-source models (e.g., Llama, Mistral, Mixtral)
  • Publicly available, customizable
  • Greater flexibility and privacy
  • Performance varies, requires local resources
• ⚖️ Trade-off: ease of use vs independence and control

Now let’s look at a key distinction that matters a lot, especially in clinical applications:
Commercial models vs Open-source models.

Commercial models — like ChatGPT, Claude, Gemini — are created and maintained by big tech companies.
They are closed-source: we can use them, but we can’t see how they work inside.
They offer very strong performance, frequent updates, and often great usability.
However, they raise privacy concerns: patient data might pass through third-party servers.
And you have limited control: you depend on the company’s infrastructure, pricing, and policies.

Open-source models — like Llama, Mistral, or Mixtral — are publicly released and can be used, fine-tuned, and even self-hosted.
They offer greater flexibility, potentially better privacy (if deployed locally), and full control over customization.
The downside is that performance may vary: smaller open models often need careful tuning.
And — importantly — they require your own hardware resources to run properly.

Bottom line:
Choosing between commercial and open depends on your priorities:
Ease of use and immediate access — or independence, customization, and privacy.

Choosing Between Commercial and Open-source LLMs

Which is better for your clinical needs?

🏥 Scenario	🚀 Recommended Approach
Rapid prototyping or brainstorming	Commercial model (easy access, strong performance)
Handling sensitive patient data	Open-source model (self-hosted, private)
Need for strong clinical language precision	Fine-tuned open-source model (customizable)
Limited local hardware/resources	Commercial model (cloud-based)
Full control over deployment and updates	Open-source model (independence)

Here’s a quick practical guide to help you decide between using a commercial model or an open-source model based on your clinical needs.

If you need to prototype fast, test ideas, or brainstorm,
commercial models like ChatGPT or Claude are very convenient: they offer strong performance right out of the box and require no local setup.

If you are dealing with sensitive patient information — where privacy and data security are non-negotiable —
an open-source model, hosted on your own secure servers, becomes a safer option.

If your goal is to achieve highly specialized clinical language understanding — for instance, generating precise medical summaries —
then fine-tuning an open-source model on clinical datasets could give you better domain-specific results.

If you lack strong local hardware (for example, no GPUs available),
commercial cloud-based solutions might be more practical.

But if you want complete control over your AI system, including updates, deployment, and custom behavior,
then open-source models are the way to go — at the cost of a little more technical management.

Ultimately, the choice depends on the balance between ease, cost, control, and risk in your specific setting.

Local Deployment Security Considerations

Protecting patient data with on-premise LLMs

• 🔒 Security advantages:
  • No data leaves institutional network
  • Complete audit trail within organization
  • No dependency on third-party privacy policies
  • Compliance with data residency requirements
• ⚠️ Implementation challenges:
  • Hardware requirements: GPU servers or clusters
  • IT support and maintenance needs
  • Model update and versioning management
  • Performance limitations vs. cloud models

Note

Consider hybrid approaches: sensitive data on local models, non-PHI on cloud models

Let’s conclude with a crucial aspect in healthcare: data security and local LLM deployment.

On-premise deployment, where models run internally within your organization, offers significant security advantages. First, sensitive patient data never leaves the institutional network - eliminating an entire layer of risk. You have complete control over the audit trail, without depending on third-party privacy policies. This also facilitates compliance with data residency requirements, increasingly common in healthcare regulations.

However, implementing LLMs locally involves important challenges. Hardware requirements are substantial - powerful GPU servers or clusters are needed, especially for larger models. Your IT team will need to support and maintain these systems, manage model updates and versioning. And there’s typically a performance gap compared to cloud models - locally executable models tend to be smaller and less powerful.

A pragmatic solution I see many organizations adopting is a hybrid approach: using local models for data containing protected patient information, and more powerful cloud models for tasks that don’t involve sensitive data.

For example, you might use a local model to analyze clinical records, but a cloud model to research medical literature or generate generic educational material.

This balanced approach allows you to leverage the best of both worlds: the security of local deployment where necessary, and the superior capabilities of cloud models where appropriate.

In the next practical module, we’ll see precisely how to configure LM Studio for local deployment and how to use WebLLM for secure executions directly in the browser.

LM Studio & WebLLM: Local LLMs for Clinical Use

Run AI models privately and offline

• 🖥️ LM Studio:
  • Desktop app for Windows, macOS, Linux
  • Download and run open-source models locally
  • Offers chat interface and API server
  • Ideal for offline, privacy-sensitive tasks
• 🌐 WebLLM:
  • Runs LLMs directly in your browser
  • No installation or backend needed
  • Powered by WebGPU for fast inference
  • Great for lightweight, portable deployments

Let’s now explore two powerful tools that allow you to run large language models locally: LM Studio and WebLLM.

LM Studio is a desktop application compatible with Windows, macOS, and Linux. It enables you to download and run open-source models directly on your machine. With its user-friendly chat interface and built-in API server, LM Studio is ideal for offline tasks where data privacy is paramount.

On the other hand, WebLLM operates entirely within your web browser. It leverages WebGPU to run models efficiently without any installation or backend infrastructure. This makes it perfect for lightweight and portable applications.

The key advantage of both tools is the full control over your data and models, eliminating the need for cloud services and enhancing privacy and security in clinical settings.

WebLLM: Running LLMs directly in your browser

A simple and private way to use AI locally

• 🌐 Runs directly in Chrome, Edge, Safari (no install)
• ⚡ Powered by WebGPU: fast local inference
• 🔒 No data leaves your computer
• 🛠️ Supports chat, document summarization, Q&A
• 🧠 Great for lightweight clinical tasks and experiments

Now let’s introduce WebLLM, a powerful but simple tool to run language models locally.

WebLLM works entirely in your browser — Chrome, Edge, or Safari.
There’s no installation needed, no backend server.
It uses WebGPU, a modern browser technology, to run models efficiently on your device.

The huge advantage is privacy:
All the processing happens locally — nothing is sent to external servers.

WebLLM supports not only basic chatting, but also document summarization and question-answering over uploaded files.

It’s a great lightweight solution for clinical environments where privacy and speed matter.

Of course, models running in-browser are smaller compared to cloud LLMs — but for many tasks, they are surprisingly effective.

How to Use WebLLM for Clinical Summaries

Simple steps to summarize clinical documents

• 🌐 Open WebLLM in your browser
• 📋 Copy and paste the clinical text into the chat
• 💬 Write this prompt:

“Summarize the key clinical information, focusing on:
- Primary diagnosis
- Treatments administered
- Follow-up instructions
- Patient condition at discharge.”

✅ The model will process the text locally and generate a summary!

WebLLM does not support uploading files like PDFs directly.

Instead, you copy the text from the document — for example, a discharge summary — and paste it into the chat interface.

After pasting the clinical text, you write a structured prompt, like: “Summarize the key clinical information, focusing on primary diagnosis, treatments administered, follow-up instructions, and patient condition at discharge.”

The model will then process the text locally and generate a private summary — without sending any data to the cloud.

Always remember to review and validate the output before clinical use!

Prompt Engineering for Clinical Applications

Techniques to improve accuracy and reliability

• 🔍 Chain-of-Thought Prompting: > “First analyze the lab values, then identify abnormalities, > then correlate with symptoms, and finally suggest possible diagnoses.”

• 📋 Few-Shot Examples: > “Example 1: Patient with [symptoms]… Diagnosis: [condition] > Now diagnose: Patient with fever, productive cough…”

• 🧩 Structured Output: > “Format your response as: Assessment: [text], Plan: [text], > Follow-up: [text], Patient Education: [text]”

Let’s discuss specific techniques to improve the accuracy of language models in clinical settings. These prompt engineering techniques are fundamental because they can make the difference between a generic response and one that’s clinically useful.

The first technique is “Chain-of-Thought Prompting” - essentially, we ask the model to show its reasoning step by step. Instead of simply asking “What’s the diagnosis?”, we ask it to first analyze lab values, then identify abnormalities, correlate them with symptoms, and only then suggest possible diagnoses. This approach significantly reduces errors because it forces the model to follow a logical diagnostic process instead of jumping to conclusions.

The second technique uses demonstrative examples, or “Few-Shot Examples.” Here we provide the model with one or more complete examples of how we’d like it to respond. This is particularly effective for specialized tasks where response format matters. For example, we can show how to correctly analyze a pneumonia case before asking it to analyze our specific case.

Finally, we have the structured output technique, where we specify exactly the format we want for the response. This is incredibly useful for clinical documentation, where we can ask the model to organize information according to standard sections like Assessment, Plan, Follow-up, and Patient Education.

In our testing, these techniques reduced hallucinations by over 60% and improved the clinical relevance of responses by nearly 40%.

Regulatory Considerations

Legal framework for AI in healthcare

• 🏛️ Key regulations:
  • HIPAA (US): Protected Health Information requirements
  • GDPR (EU): Special category data processing restrictions
  • MDR (EU): AI as medical device classification
• ⚖️ Compliance challenges:
  • Data residency requirements for processing PHI
  • Right to explanation for AI-assisted decisions
  • Audit trails for AI-generated content

Warning

Always check if your LLM usage requires: 1. Patient consent 2. Data processing agreements 3. Classification as a medical device

This slide introduces the key regulatory frameworks that apply to healthcare AI applications.

For HIPAA compliance, the key concern is whether PHI is being transmitted to third-party servers when using commercial models. This is why on-premise models are often necessary for clinical applications.

Under GDPR, health data is considered “special category data” with additional protections. AI systems must be transparent, and patients generally have the right to know when AI is being used and how it affects decisions.

The Medical Device Regulation in Europe and similar FDA frameworks in the US may classify certain AI applications as medical devices requiring certification, especially if they’re used for diagnosis or treatment planning.

These regulatory issues directly impact decisions about which models to use and how to deploy them.

Ethical Documentation Requirements

Transparency in AI-assisted clinical notes

• 📝 Best practices:
  • Disclose AI assistance in documentation
  • Specify which parts were AI-generated
  • Document human verification steps
  • Maintain separation of AI suggestions and clinical judgment

• ✅ Example disclosure: > “This assessment summary was drafted using AI assistance and reviewed by Dr. Johnson for accuracy. All interpretations and medical decisions were independently verified.”

This slide addresses the ethical requirements for transparency when using AI to assist with clinical documentation.

There’s growing consensus among medical ethics boards that patients have a right to know when AI tools were used in their care, especially for documentation and decision support.

The disclosure doesn’t need to be extensive, but should acknowledge where AI was used and emphasize that human clinical judgment remained central to the process.

This transparency builds trust, establishes appropriate expectations, and creates a foundation for ethical AI use in healthcare settings.

Liability Considerations

Managing risk when using LLMs in clinical settings

• ⚠️ Current legal landscape:
  • No clear precedent for AI liability in healthcare
  • Professional standards still evolving
  • Default position: clinician bears ultimate responsibility
• 🛡️ Risk mitigation strategies:
  • Document verification procedures for AI outputs
  • Establish clear workflows for critical vs. non-critical uses
  • Train staff on limitations and verification requirements
  • Maintain awareness of model-specific limitations

Note

Consider consulting with risk management and legal counsel before implementing LLMs for clinical decision support.

This slide discusses the evolving landscape of liability when using AI tools in healthcare.

The key point is that regardless of how advanced AI becomes, the clinical professional remains responsible for verification and decisions. This means having clear processes for checking AI outputs before incorporating them into clinical care.

The risk level varies by application type: using AI to help draft patient education materials carries less risk than using it to interpret lab results or suggest diagnoses.

Healthcare organizations should develop clear policies that specify which applications are permitted, what verification steps are required, and who bears responsibility at each stage.

This is still an evolving area legally, but maintaining clear documentation and verification steps helps establish that the standard of care was met.

Cost-Benefit Analysis of LLMs in Clinical Settings

ROI considerations for medical implementations

• 💰 Cost factors:
  • API costs: $0.50-$20 per 1,000 clinical notes (model dependent)
  • Staff time saved: 20-40% reduction in documentation time
  • Training & implementation: 40-80 hours per department
• 📊 Real-world ROI examples:
  • Hospital A: 50% reduction in discharge summary time (saving 15 min/patient)
  • Clinic B: 30% increase in note completeness and quality
  • Lab C: 70% faster protocol drafting for new tests

Note

ROI is typically reached within 3-6 months when focusing on high-volume documentation tasks.

Let’s move to the practical aspects of implementing LLMs in clinical settings - specifically, cost-benefit analysis. Hospital directors and laboratory managers constantly ask me: “Are these tools worth the investment?”

Let’s examine the main cost factors. API costs vary significantly depending on the model - we can spend from 50 cents up to $20 to process 1,000 clinical notes. This depends on the complexity of the model and the provider.

The main savings come from staff time. We’re seeing 20-40% reductions in time spent on documentation. Think about what this means for doctors and specialists who spend hours each day writing notes.

Implementation typically requires between 40 and 80 hours per department, including training and integration into workflows.

But let’s look at some concrete examples. A hospital we collaborate with reduced discharge note writing time by 50% - that’s 15 minutes saved per patient. Multiply that by hundreds of patients per month.

A clinic reported a 30% increase in the completeness and quality of notes, improving clinical communication and reducing clarification requests.

A laboratory accelerated protocol drafting for new tests by 70%, allowing faster implementations.

In most cases, return on investment (ROI) is achieved within 3-6 months, especially when focusing on high-volume activities like routine documentation.

But remember: the value goes beyond simple time savings. There’s also a qualitative improvement in documentation that can positively influence patient care.