We are not the only ones who have come across the problem of choosing which LLM to use as a company building digital patient twins for randomised control trials. This is a critical decision for founders, developers, and product managers especially in healthcare. Reason being, healthcare applications demand higher levels of accuracy, data privacy, and domain expertise, so selecting the right model (or models) requires careful consideration.
The LLM landscape in 2025 offers a mix of proprietary services (OpenAI, Google, Anthropic, etc.) and open-source models (Meta’s LLaMA 2, the new Mistral, etc.), each with strengths and trade-offs. In this comprehensive guide, we’ll explore the LLM options available, key factors to weigh for healthcare use cases, and highlight a real-world use case example of ourselves and products to illustrate how the choice of model can enable innovative solutions in clinical trials.
Today’s LLM options fall broadly into two categories: proprietary cloud-based models offered as services, and open-source models that you can run or fine-tune yourself. Understanding the differences is a first step to narrowing your choices;
1. Proprietary LLM Platforms (Closed-Source)
These are large models developed by organizations like OpenAI, Google, and Anthropic, offered via APIs or cloud platforms. You don’t get direct access to the model weights, but you benefit from a fully managed service and cutting-edge capabilities.
Key players include:
● OpenAI (GPT series): OpenAI’s GPT-5 is widely regarded as the state-of-the-art general LLM as of 2025, excelling in reasoning and
knowledge across domains. This near-expert medical proficiency gives GPT-5 a strong edge for complex healthcare applications. The
downside is cost and the need to send data to an external API, which raises data privacy considerations.
● Google (PaLM 2 and Med-PaLM 2) now integrated with Gemini: Google’s flagship LLM is PaLM 2, accessible via Google Cloud’s
Vertex AI platform. Notably, Google developed Med-PaLM 2, a version tuned specifically on medical knowledge. This can now be more
broadly accessed on google Gemini as Gemini as these models are designed to be multimodal, allowing them to process and create
diverse content, including text, images, video, and code.
● Anthropic (Claude 2): Anthropic’s Claude 2 is another top-tier model known for its emphasis on safety and long-context handling.
Claude was built with “Constitutional AI” techniques to minimize harmful or biased outputs, making it attractive for healthcare use where
harmlessness and clarity are crucial.
2. Open-Source LLMs (Self-Hosted or Fine-Tunable)
On the other side of the spectrum are open-source or openly licensed models. These can be run on your own servers or modified to fit your needs. For teams building healthcare AI products, open models offer full control over data and customization, often at lower incremental cost but with the trade-off that you must handle deployment and may need to sacrifice some raw performance compared to the very largest closed models.
Leading open-source models include:
● Meta LLaMA 2: LLaMA 2 opened the door for many developers to experiment with a high-quality base model under a permissive
license. However, the open-source advantage is that you can fine-tune LLaMA 2 on your own medical text data or adjust its behavior,
something not possible with closed APIs. Many in the community have created fine-tuned variants (for coding, chat, etc.), and you could
fine-tune it further on clinical notes, guidelines, or Q&A pairs to infuse medical expertise.
● Mistral 7B: A model like Mistral 7B can be run with much lower hardware requirements while still delivering strong results in
understanding and generating text. The Mistral team released it under the Apache 2.0 license (very permissive), explicitly allowing
commercial use. In practical terms, Mistral 7B might handle many conversational and reasoning tasks needed for healthcare
applications (like triage Q&As or summarizing a patient note) at a fraction of the cost of GPT-5. It may struggle more with highly
specialized medical knowledge compared to a larger model, but it can always be fine-tuned further on medical data.
3. Other Open Models:
In addition to LLaMA and Mistral, there are numerous other open-source LLMs that might be relevant:
● Domain-Specific Models: There are emerging efforts to create medically specialized open models by fine-tuning on biomedical
literature. For example, BioGPT (by Microsoft Research) and PubMedGPT are trained on biomedical papers and aim to better handle
medical terminology. However, these are often smaller or less general than the big mainstream models.
Key Factors When Choosing an LLM in Healthcare
Not all healthcare AI products have the same requirements. Here we break down the key considerations you should evaluate when picking an LLM for your project:
1. Accuracy and Domain Knowledge: In healthcare, getting the facts right is paramount – an AI that fabricates a condition or gives a
wrong dosage advice can be dangerous. You’ll want a model with demonstrated strong performance in medical reasoning and
knowledge. GPT-4 has proven exceptional on medical exam questions (scoring in the 90%+ range), and Google’s Gemini reached
expert-doctor level on USMLE questions. Full disclosure we use both Gemini and GPT-5.
2. Privacy and Data Security: Healthcare data (PHI) is highly sensitive and regulated. This is often the make-or-break factor in choosing
an LLM. Using a cloud API means patient data leaves your environment you must ensure this complies with regulations and your
internal policies. Providers like OpenAI and Google do offer business agreements (and in some cases HIPAA BAA support) so that data
isn’t used for training and is handled securely, but some organizations are still uncomfortable with any external data transmission. If that
is a concern, an open-source model that you can deploy on-premises or in a private cloud is a better choice for you. By self-hosting,
you keep all data within your firewall and have full control over encryption, access logs, etc. The trade-off is the burden of managing the
infrastructure and updating yourself.
3. Cost and Scalability: LLMs can rack up costs, either in API fees or compute infrastructure. Proprietary models typically charge per
token. OpenAI’s top model might cost 10× or more compared to using a smaller model. We’ve found that using GPT-5 for a text
summarization task was 18× more expensive than running a comparable gemini model, for roughly similar output quality. Estimate your
usage (tokens per month) and get quotes for both API and self-hosted infrastructure to compare. Also consider scalability: can the
solution handle peak loads? Cloud APIs abstract that for you (auto-scaling on OpenAI’s side), whereas with an open model you’d need
to provision enough servers to meet your highest demand.
4. Fine-Tuning and Customization: Out-of-the-box LLMs are generalists. Healthcare products often need the model to speak a certain
way or know particular content. For example, a patient-facing chatbot should have a compassionate tone and stick to layman’s terms; a
clinical decision support assistant might need to cite specific hospital guidelines or use internal medical terminology. If you anticipate
needing such customization, check what each option allows:
○ Open-source models are fully customizable, you can fine-tune the model on your own dataset of dialogues or documents, so it
learns the patterns you want. You could, for instance, fine-tune a model on a set of verified Q&A pairs about your hospital’s procedures or on de-identified EHR notes to teach it clinical language. Fine-tuning does require having training data and computational resources, but libraries like Hugging Face Transformers have made the process fairly straightforward for smaller models.
○ Proprietary models historically did not allow fine-tuning (you got what you got), but this is changing. Other providers like Cohere
also allow fine-tuning on their base models. The fine-tune is done on the provider side (you upload data, they train a custom
model for you). This can significantly improve performance on specialized tasks while keeping the convenience of an API. The
downsides: it can be expensive, and you are entrusting your fine-tuning data to the provider (which could be a concern if the
data is sensitive).
○ If not full fine-tuning, consider prompt engineering or few-shot prompting as lighter ways to customize. Closed models let you
supply a system prompt each time (e.g., “You are an empathetic medical assistant...” plus some instructions). That can shape
the style and some behavior without any training. Few-shot prompting (giving a couple of QA examples in the prompt) can guide
the model to use a desired format. These techniques, however, have limits and make each request longer (more tokens).
5. Reliability and Safety:
In healthcare, it’s not just about what the model can do, but whether it does it safely and consistently. This includes avoiding dangerous
errors (e.g. hallucinating a medication recommendation that could harm) and maintaining a respectful, non-biased tone with patients.
Closed models like GPT-4 and Claude have undergone a lot of safety training and red-teaming. For example, Claude 2 was evaluated
to be 2× better at giving harmless responses compared to its previous version. It also refuses tasks that involve medical advice beyond
its scope, etc. OpenAI’s models have content filters (which can be a blessing preventing disallowed content but also a curse if they
unnecessarily block some legitimate medical discussions). Open-source models, unless you fine-tune or add a filter, will respond with
anything they were trained on they may lack built-in guardrails. If you go the open route, you will likely need to implement a moderation
layer yourself (OpenAI provides a moderation endpoint; or use rule-based filtering on the outputs). This is particularly important for
patient-facing applications: you wouldn’t want the AI to use overly blunt language or expose private info to the wrong user.
● Integration and Ecosystem: Finally, consider the practical aspects of integrating the LLM into your product. Proprietary APIs have
well-documented SDKs, and many third-party libraries support them. For instance, OpenAI’s API can plug into open-source frameworks
like LangChain or LlamaIndex which simplify building chat or retrieval-augmented systems. If you use Azure’s or Google’s services, they
integrate with other cloud services (storage, databases, etc.) which might ease development if you’re already on that platform. On the
open side, tools like Hugging Face’s transformers and text-generation-inference can serve models with high performance.
There are also managed solutions (like Hugging Face Hub or cloud marketplaces) that can host open models for you, acting like an API.
These could be a middle ground: you get an open model with a vendor handling the scaling. That said, not all open models will have a
turnkey solution; you may need in-house ML engineers. Support is another angle closed providers usually offer enterprise support
plans, so if something goes wrong, you have recourse. With an open model, your “support” is community forums or your own team.
Depending on your startup’s expertise, one or the other will feel more comfortable.
Case Study: Infiuss Health’s Digital Patient Twins for Randomised Clinical Trials
To ground this discussion,I will give a sneak peak into our own product experiments. At Infiuss Health, we are focusing on using digital patient twins to help simulate real patient responses before commencing expensive randomized controlled trials (RCTs).
What are digital patient twins? A digital patient twin(DPT) is a highly detailed computational replica of a real patient, a “virtual twin” that
mirrors the person’s physiology, medical history, and potentially how they respond to treatments. These twins are built using AI models trained on multi-omic clinical datasets, so that they behave realistically as a real patient would. Infiuss’s platform uses digital twins as a way to simulate patient responses to RCT protocols and to optimise the study protocol.
What are RCTs? A Randomized Controlled Trial (RCT) is a type of scientific study used to test whether a new treatment, drug, or healthcare intervention actually works.The key component of RCTs is that participants are randomly assigned to one of two or more groups:
● Intervention group(s): receive the treatment or intervention being tested.
● Control group: receive a placebo, standard treatment, or no intervention at all.
Because the assignment is random, researchers can be confident that differences in outcomes between groups are caused by the treatment itself rather than other factors. Despite being considered the “gold standard” for testing new treatments in medicine because they minimize bias, allow for clear comparisons, and generate high-quality evidence on safety and effectiveness they are notoriously hard to run. Recruiting patients, predicting enrollment rates, and dealing with patient dropout can derail trials with nearly 80% of trials fail to meet enrollment targets on time, and many run over budget.
How do LLMs or AI models come into play here? Building a faithful digital twin is a complex AI task: it requires modeling disease
progression, treatment effects, and patient variability. LLMs (or more broadly, generative models) can be used to predict patient trajectories and even generate realistic clinical notes or responses.
At Infiuss we use a combination of data-driven modeling and LLM-like simulation. We start by aggregating multi-layer data de-identified
electronic health records, genomic and lab data, wearable sensor data into a large repository. This data is then cleaned and stratified to
generate a set of statistically faithful digital patient profiles that capture the real-world diversity of patients. In other words, the AI ensures that the virtual patients have characteristics and variability similar to actual patients (different ages, comorbidities, etc., reflecting real population heterogeneity).
This approach helps in stratifying cohorts and making informed decisions. For instance, the simulation might reveal that a certain eligibility criteria is unnecessarily excluding many patients, and relaxing that (while still safe) could speed up enrollment. Or it might show that patients with a certain comorbidity have a higher chance of adverse events, prompting closer monitoring for that subgroup.
How we select the right LLMs to use: Our digital twin use case is a bit different from a standard chatbot as it involves generative modeling of patient data. Because we needed models that could handle multimodal data (clinical text, structured data, etc.) and generate realistic outputs. We opted to use llms to simulate things like patient-reported outcomes or to generate plausible medical histories. When choosing models for such a system,our team had to consider:
● Accuracy of medical representation: To generate a patient’s symptom description or response, it needed to be grounded in medical
reality ( fine-tuned on real patient records).
● Ability to incorporate clinical data: We needed models that could take structured clinical input (e.g., lab results) alongside text. This
could argue for models that allow conditioning on structured data or using techniques like prompt templating or tool use.
● Privacy: Since use real patient data to build the twins, those data likely stay in-house.
● Efficiency: Hundreds of simulations mean the model needed to run many times using a very large model might be cost-prohibitive. We
use smaller specialized models for each aspect (one model for simulating physiological responses, another for simulating patient
dropout, etc.). Each of those could be an open-source model fine-tuned on specific tasks (for example, a model fine-tuned to predict trial
enrollment based on criteria). So obviously, one model could not work for us. Below if out LLM stack;
1) Data & Retrieval Layer
● Data lake: de-identified EHR, labs, omics, wearable streams, site ops data, protocol metadata.
Document stores: trial protocols, SOPs, eligibility criteria, CRFs, monitoring plans, country-specific recruitment guides.
Embeddings
○ Primary (PHI-safe, on-prem): bge-large-en or GTE-large for English clinical text.
○ Secondary (non-PHI, cloud): OpenAI text-embedding-3-large for public docs if you need cross-vendor parity.
● Vector DB: pgvector on Postgres for simplicity or Milvus for scale.
● RAG policies
○ Strict chunking of protocols and guidance.
○ Source-citation required for any LLM answer surfaced to users or used for ops decisions.
2) Model Layer (two tiers)
Tier A — On-prem, privacy-first
● LLaMA-2-70B-Chat fine-tuned on de-identified notes and protocol language for:
○ Eligibility parsing, cohort stratification suggestions, ops Q&A over internal docs.
○ Patient-narrative synthesis for twin realism and adverse event scenario drafting.
● Mistral-7B-Instruct for high-throughput tasks:
○ Data normalization explanations, templated site emails, shortform summaries, CAPA draft suggestions.
● Task models
○ Clinical de-identification: BiLSTM-CRF or RoBERTa-deid pipeline in front of any cloud calls.
○ Tabular reasoning: LightGBM or XGBoost for enrollment forecast features that the LLM explains.
○ Time series simulators sit outside the LLMs and feed text to them for narrative outputs.
Whichever path you choose, always keep in mind the responsibility that comes with deploying AI in healthcare.
Happy building