Trust, but verify: ChatGPT can be a powerful tool, but also one that leads students and researchers astray
ø
Librarians, teachers, and students beware! ChatGPT can be prone to wildly wrong answers. Treat ChatGPT as a tool and resource that needs additional vetting.
I have already documented problems with ChatGPT answers about STEM subjects and issues. I have found many answers to be antiquated or errant. One reason for these errors is because the ChatGPT knowledge base is akin to an inverted pyramid. Distant events should enjoy a wider base of texts upon which to train large language models. Documentation of recent discoveries or events is usually less, therefore rendering ansers more susceptible to single -source errors or outdated material.
While doing genealogy research last week on Adams and Boylston ancestors (e.g., one of my 7th great-grandfathers was Peter Boylston Adams (1738-1823), the younger brother of President John Adams Jr., John Adams Sr is an 8th great grandfather, and Susanna Boylston is an 8th great-grandmother) I decided to see how ChatGPT performed outside STEM in an area where the base of historical knowledge available is both wide and deeply vetted.
I was shocked at its failures
Things ChatGPT ERRANTLY claims about Peter Boylston Adams (1738-1823) included that he was:
(1) the father of The 2nd U.S. President John Adams (who was actually his older brother);
(2) the grandfather of Abigail Adams;
(3) a lieutenant in the Continental Army (he was A Captain in the Mass Militia);
(4) the grandfather of President John Quincy Adams.
None of that is true.
When I hit regenerate, ChatGPT “apologized” for its prior errors but then made still wilder mistakes by errantly claiming the that Peter Boylston Adams (1738-1823):
(5) was a second cousin of President John Adams (he was his younger brother);
(6) was a sea captain and merchant;
(7) was a captain in the Continental Navy;
(8 ) as a sea captain was twice captured by British ships but that managed to escape both times.
Again, none of that is true.
When I specifically asked if Captain Peter Boylston Adams (1738-1823) was the brother of John Adams, ChatGPT said “no” and doubled down on its prior false answers.
Three strikes and you’re out ChatGPT!
These are not trivial errors. Many correct genealogies for the Adams family are available online and in physical library collections. These are also the type of errors, set in seemingly authoritative paragraphs, that lead students and researchers astray.
