#1
3 Likes
#2

IMO, claiming a state of “surpassing” is inappropriate given than an LLM extracts and organizes 100% of its knowledge from the extant published work of physicians and medical scientists. What LLM’s do well, and what has aided me greatly in my personal areas of research, is keeping track of a large number of potentially relevant variables, ranking them for their potential contribution to the question or problem, and suggesting possible solutions and/or further lines of inquiry.

Any reasonably well-trained and skilled physician could do a much better job of diagnosing if they had (or could take) the time to look up and consider the relevant research and build a fact assembly chart or, these days, to spend 15 minutes in an interactive discussion with an LLM. The problem is not so much a lack of competence on the part of the physician as it is that the current service and diagnostic model requires determining the problem and effecting its solution within a 12 minute block of time that must also include time for a greeting and time for writing a prescription. Unfortunately, these time limitations apply to rarefied specialists as well. I recently observed this problem when a friend, himself a physician, had to beg for additional time with a specialist of reputation who insisted on imposing a quick solution that we knew was likely ignoring some of the evidence presented to him. Eventually, the physician in need of treatment and I arrived at a correct diagnosis and treatment between us using LLMs as time-saving research tools.

It is definitely a new frontier!

Parenthetically, this kind of hyperbole is Musk 101. He guaranteed that fully autonomous cars would be in the road in 2018 and now promises to reduce two trillion dollars from the US budget, a mathematical impossibility without eliminating the entire military, defense, and intelligence budgets, plus a handful of other smaller essentials. Moreover, based on federal spending indexed on GDP, the US government is the most efficient of all of the large advanced nations, except China in which the absence of democratic processes and standards of living lower costs significantly. Looking for and eliminating unneeded inefficiencies in government is always a good idea but not if it is done on a partisan basis.

4 Likes
#3

I wouldn’t personally send much of my data to OpenAI or other LLM’s online unless in very specific scenarios.

However this could be a general thought out approach:

  1. Take blood tests and other measurements. Other data and important information, lifestyle, habits, as necessary.
  2. Download the best locally run model, i.e QWEN 2.5 at the moment. Upgrade hardware if necessary (speed of inference is determined by memory bandwidth, either VRAM or RAM).
  3. Find all of the measurements that are outside optimal range, ask model how to move biomarker to optimal range, further tests, and other, could use prompting like as if it’s a doctor “the patient has X biomarker at X level, how to lower?” or other prompt engineering.
  4. Implement and then repeat 3.

So in a few years the models will be better than any regular doctor for optimizing health.

1 Like
#4

The researchers said they were surprised at how well Chat GPT Plus alone performed, with a median diagnostic accuracy of more than 92%. They say this may reflect the prompts used in the study, suggesting that physicians likely will benefit from training on how to use prompts effectively. Alternately, they say, healthcare organizations could purchase predefined prompts to implement in clinical workflow and documentation.

The researchers also caution that Chat GPT Plus likely would fare less well in real life, where many other aspects of clinical reasoning come into play — especially in determining downstream effects of diagnoses and treatment decisions. They’re urging additional studies to assess large language models’ abilities in those areas and are conducting a similar study on management decision-making.

“As AI becomes more embedded in healthcare, it’s essential to understand how we can leverage these tools to improve patient care and the physician experience,” Parsons said. “This study suggests there is much work to be done in terms of optimizing our partnership with AI in the clinical environment.”

2 Likes
#5

I highly doubt it…
Let’s start with a typical doctor visit.

Subjective: Why is the patient here. Patient rarely ever present their symptoms clearly. There is a lot of non-verbal communication. Certain symptoms are emphasized. Persistent symptoms are de-emphasized (but I also wheezed since I was a child.) Tone of voice is important. Observing the patient is a good part of this, is the patient in distress? Do they appear anxious? Is the real reason why they are here or they are purposely beating around the bush.

None of those subtle communication bits will be picked up by AI today or in near future.

Objective - Can AI perform a physical exam. Can it smell a foul breath odor of a sinus infection ? Can differentiate a stridor from a wheeze? Can it feel the the skin lesion to see if it’s a patch lesion, wheal or macule ? Can it make it blanch ?

Assessment I think this may be the strength of AI, that once the subjective and objective data is inputted it can come up with a differential diagnosis and suggest further action - this can augment the physician for sure. However most conditions are not black and white. There is a lot of overlap. Sometimes you have to pick your battles too, what do we address first. What is most important clinically and to the patient.

Plan How will AI ensure patient’s compliance ? Does understand that this busy mom working two jobs will not use this recommended medication 4 times a day. Does it understand that a teenager will not check his peak flows to determine need for albuterol and best you can do is get them to use the maintenance inhaler once a day. Can it think about out of the box with unusual presentation of common condition or a very unusual condition that has no standard of care.

Can AI actually perform the ART of medicine ? Will it show empathy and realize that sometimes the patient just needs to be listened to and their complaints validated.

I doubt it.

9 Likes
#6

I don’t have the reference but there was a recent study looking LLM in 3 aspects of patient care and the LLM failed all 3.

3 Likes
#7

Just curiosity, would you send your data anonymously should they give you the chance?.

#8

It’s simpler to do it all locally. But no as I wouldn’t believe that it would be anonymous in the first place and the company can store all of your prompts and data. But I can see circumstances where you would use one.

#9

I did see something recently that implied any information you enter into chatGPT essentially becomes public information.

#10

For what’s it’s worth, when I was building prompts for Gpt4o I practiced with it’s voice features. The newest iteration was able to tell, when I purposely became agitated, that I was agitated telling me to ‘slow down’, ‘breathe’ and take a minute to collect my thoughts.

2 Likes
#11

I’m fortunate that in the ER a fair amount of what I do is procedures requiring use of my hands. So, I’ll probably be able to retire before AI takes over. But, the robots are coming.

1 Like
#12

He also had SpaceX thats outperforming anything NASA or Boeing have. Did you ever think after a launch the bottom part of the launch would be able to land on a platform so it could be used again in a short time, saving millions. Now he was able to catch the capsule on the launch pad. Lets give him and Vivec a chance to see how much government waste they can eliminate. Companies wouldn’t be spending Billions on AI if they didn’t think it could eventually solve problems and a lot quicker.

#13

I’ll skip the political dimensions of this comment to note that framing NASA as somehow in competition with private entities such as SpaceX misunderstands NASA’s long range strategic plans for the benefit of the nation, plans that have been in place and on which they have executed for decades.

NASA’s objective in this regard is to advance the early phases of America’s position in the frontiers of space where the costs ROI, and risk/reward coefficients of doing so lie beyond the capabilities of the private sector. Then, as NASA is successful in rationalizing specific enterprises, their objective becomes identifying ways to move it successfully into the private sector while maintaining a level of executive oversight appropriate to ensuring taxpayer’s interests. Thus, the successes of SpaceX in this instance reflect well not only on the company but on NASA’s judgment in framing outsourcing, awarding bids, and maintaining oversight on these very large initiatives to ensure that they continue to align with NASA and taxpayer interests. Another way of putting it is that NASA’s goals are to outsource functions once they reach a size and manageability that one can reasonably expect that the private sector can execute on them in the proverbial cheaper, faster, and better way. On balance, these managed contracts have served us well, especially given the impossibility of avoiding mistakes in the early phases of any scientific and technological inquiry.

Virtually all of Mr. Musk’s SpaceX and much other revenue and profits have come directly from taxpayers via managed contracts.

1 Like