I have been researching the use of LLMs in a variety of settings for the past two or three years. Note that these are locally-hosted systems, not Saas, thus ensuring sensitive data is kept in-house and experiments can be, er, extensive.
It's my view that such systems can be useful in instances where their reach is bounded. By that I mean that if you have a specific query, or specify a text that you want to analyse,
and you also specify a set of source documents to use with said analysis or query, and you are very precise in your prompting, then they can produce some quite timely and useful results. In other tests I have found them to have mixed outcomes with general coding, and with analysis of handwritten documents, and generally positive or amusing outcomes when asked to write a story (eg. "write a 10 page story about Darcy and Elizabeth's great-grandchildren's first motor vehicle crash"). The latter is assisted if you give some background documents on early English motoring...
But, as the LLM sphere is at present, I would most definitely
NOT use them for a general, unbounded, query engine! I'm entirely unsurprised by the OP's relative's result, amused by some of the responses here, and heartened by others, so perhaps not all is lost, however I have seen posts on this site where people have presented the results of a
stochastic parrot as authoritative - which I find disturbing, particularly when they come from 'senior' members.
Takeaway: if you want to know the answer to a general knowledge question, try going to the fundamental source, then Wikipedia perhaps where at least there's usually some peer-review process. @TasmanAir is particularly on the point about lack of validation, and perhaps that will come in time (despite sounding negative I'm not, the year-on-year advancements are astounding - see the
2025 and
2026 Stanford AI index report for example), but they're definitely not there yet.
FP.