Earning the right to act: building specialized clinical AI for women's hormone health

Earning the right to act: building specialized clinical AI for women's hormone health

Every few weeks a new frontier model arrives that scores higher on medical exams than the last one. If OpenAI, Anthropic and Google keep getting better at medicine, what does specialization mean? And what makes medicine different from other applications of AI?

At Dama Health, we are building the clinical AI for women's hormone health. Today, it reaches users in two ways. Dama Assist is our clinical decision support tool for any clinician looking after the health of women, and our API is an intelligence layer for platforms that want to prescribe hormonal medications with personalization at scale. Clinicians reach for it when deciding on a medication, the exact dose and regimen, switching a formulation, weighing the benefits and risks of a treatment plan, or prescribing for a patient whose history doesn't fit any protocol or guideline. Over time, this intelligence can enable automation and wider access to care.

Lessons from leading AI companies

Most specialized AI companies aren't trying to compete with the frontier labs on intelligence. What they build is a harness: the agentic workflows, context, tools and usage data that turn a general model into something that does a specific job very well. The important part of this is the right benchmark. Harvey built BigLaw Bench and it has become the gold standard legal benchmark for the whole industry, so a specialist's benchmark can even help set the bar for the frontier labs. As frontier models caught up on BigLaw Bench, Harvey went multi-model, routing each task to whichever model handled it best, and later trained its own legal model on an open-weight base. Cursor took a similar route in coding. It built CursorBench from real developer sessions, because public coding benchmarks didn't reflect what developers actually ask agents to do, and pairs it with live evaluation on real usage.

What holds value as frontier labs get better and better? Specific workflow, usage data and specialized benchmark.

Medicine requires more than intelligence

A model can score better than doctors on medical exams and still have no permission to act: to prescribe a medication, order a lab or a scan, or refer a patient. Doctors don’t earn that permission through exams alone either. We spend years in supervised training, building clinical judgment, learning when to escalate, and taking on more responsibility as others come to trust us with it. That is the practice of medicine, and it comes with liability, responsibility and ownership of the outcome.

For clinical AI, permission to act will be earned one task and indication at a time, much as FDA approvals are, and each company will need its own clinical evidence, insurance and monitoring for its area. So in clinical AI, two questions sit alongside any benchmark score. Initially, can the AI actually be used inside a healthcare system, by busy clinicians, with real patients? And then, with enough clinical outcome evidence, is the company set up to take responsibility for the workflow it proposes?

A company that has specialized in a specific clinical area can, and that's the role we are building Dama to take in women's hormone health. This is also why we evaluate our AI systems critically in two ways: against a specialized benchmark and in real-life clinical workflows.

The Dama Hormone Health Benchmark

To hold ourselves and our product to the highest standard, and to know it's working as intended, we needed a rigorous way to evaluate it.

Because we work with experts in women's health, we have access to the latest evidence and modern clinical consensus. That means we can build case sets, evals and rubrics, and judge AI against that standard. There is no other medical benchmark that tests AI specifically on hormone health and hormone prescribing, so we created the Dama Hormone Health Bench.

Our evaluations don't just test whether AI can answer questions from a menopause guideline. Our cases are complex and drawn from the day-to-day challenges of our users: medication titration, formulation switches, hypo- and hyper-absorbers, testosterone for women, nuanced benefit and risk assessments, contraception in complex patients and therapy in high-risk profiles. Each question comes with its own criteria for what a correct answer must include and what an incorrect one gets wrong, and the set keeps growing.

As of November 8th, across >60 expert-curated clinical cases and questions, Dama Assist ranked first of the 12 configurations we tested, averaging 3.19 out of 4 on accuracy and completeness. The best general model, Claude Opus 5.5, scored 2.80.

We ran Dama Assist against the latest models from Anthropic, OpenAI and Google, with and without web search. Every configuration answered every question three times, and three AI judges from three different labs graded each answer against the clinicians' criteria: 2,196 answers and 6,588 grades. Each judge leans towards its own model family, so we average all three.

Each judge scores every answer on four dimensions: accuracy, completeness, clinical usefulness and clinical safety. Safety is binary on purpose: an answer is either safe to act on or it isn't, and there is no partial credit for a recommendation that could harm a patient.

Each of the three judges, grading alone, ranked Dama Assist first. A lot of it came from completeness: Dama's answers covered more of what experts said must be there (3.11 against 2.60). They also made fewer errors (accuracy 3.27 against 2.99), and 95.8% were rated safe, compared with 92.3% for Opus 5.5.

That matters, because omission is one of the most common, risky and least visible errors in clinical AI. A hallucination can often be caught because something in the answer is wrong. An omission is harder to detect, because nothing looks wrong: the clinician has to know what should have been there.

The result I find most telling is what happens to the same base model with and without Dama around it. Our harness lifted Gemini 4 Argon by 0.59 points, while adding web search to general models barely moved them. With the same harness on Gemini 3.8 Flash, a smaller, lower-cost model, Dama Assist scored 2.92, above every general-purpose configuration we tested; against Claude Opus 5.5, the strongest of them, the gap was within the margin of error. That configuration also scored highest of all 12 on clinical usefulness (3.79), with 95.6% of answers rated safe.

This means that for a well-defined clinical area, you don't always need the most expensive model to reach high performance and a good clinical standard. This matters tremendously when we think about cost-effectiveness in healthcare. And because our specialist layer sits on top of whichever model is best, Dama Assist gets better each time the frontier does.

Beyond the benchmark: clinicians and real workflows

A benchmark is a bit like a medical school exam or a driving test. You need to pass it, and passing doesn't automatically make you an excellent doctor or an excellent driver. What we want to know is the impact of using Dama Assist in a clinical workflow: whether it makes clinicians more confident, supports access to care, and fits into the few minutes a clinician has between appointments.

Dama Assist inside a US health system

Menopause care in the US is increasingly falling to primary care, and primary care was never trained for it. In a survey of US residents in family medicine, internal medicine and obstetrics and gynecology, only 6.8% felt adequately prepared to manage menopause. Since June 2026, internal medicine physicians at Women's Integrated Services in Health (WISH) at the University of Colorado Anschutz and UCHealth have used Dama Assist in day-to-day care, in a pilot led by WISH's medical director and her team. They brought it the hard cases: safety checks and contraindications, side effects and bleeding, lab interpretation, contraceptive prescribing in specific conditions, testosterone and perimenopause. By mid-October they had asked it nearly 1,800 questions. Nine in ten of the clinicians said Dama Assist was more useful than the AI tools they already had, and 70% said it had saved them from seeking a second opinion, whether a referral, a colleague or a separate search. For a health system, that second number is capacity and access. Each avoided second opinion is a visit completed with confidence, and a patient treated in the room instead of joining a specialist waiting list.

Independent Review from Nurse Practitioners.

Nurse Practitioners are also leading the way in meeting the demand for women’s hormone care. In September, Dama Assist earned the NurseApproved "Tested & Trusted" Seal. NurseApproved is a nurse-led review body, and its panel here was eight nurse practitioners specializing in women's health, hormone therapy and menopause care, with 124 years of clinical experience between them. Dama Assist scored 92% on their quality index, which covers clinical validity, usability, safety, confidence and workflow fit, and all eight (100%) said it works better than OpenEvidence, UpToDate or general AI for female hormone and menopause care. One reviewer's comment stays with me: "Dama actually understands the difference between progestin versus progesterone!" It sounds like a small thing. It's exactly the kind of distinction general tools blur when they treat "hormone therapy" as one category, and it's the reason so many clinicians in our field told us they had stopped using general chatbots for menopause care.

What comes next

These are the first results we're sharing from a process we have been running internally. We’ll be expanding the benchmark and publishing results as new models come out.

This work is going to take a village. It will take engineers, scientists and clinicians working together. We'd love to hear from:

• AI labs and model builders who want to know how their models perform in women's hormone health, and to test them against our benchmark with us.
• Health systems, clinics and digital health platforms whose clinicians or members need hormone care, and who want to pilot Dama Assist or build on our API.
• Menopause and women's health societies and researchers who want to co-publish the benchmark, or help agree standardized outcome measures for hormone care that any AI tool can be tested against.

We’ve shared the headline results here. For the full benchmark report, including scores on every dimension, confidence intervals and how we validated our AI judges, get in touch with the team: support@damahealth.com

Dama exists to expand access to personalized hormone care, so every woman can get a personalized treatment matched to her needs, lifestyle and biology. If any of this is relevant to your work, get in touch. And if you're a clinician, you can try Dama Assist at damaassist.com.

Sources

• Dama Health, Dama's Hormone Health LLM Benchmark V1 Technical Report (November 2026)
• Karam et al., Menopause, 2026: general LLM accuracy on menopause and hormone therapy questions
• Harvey: Introducing BigLaw Bench
• Harvey: Expanding BigLaw Bench
• Cursor: How we compare model quality in Cursor
• Paulina Cecula, Who signs the prescription? The real question in clinical AI (Substack, October 2026)
• Paulina Cecula, Are we evaluating LLMs properly, especially for women's health? (Substack, March 2026)