Is Pangram AI Detector Accurate? We Tested It
Pangram Labs calls itself "an AI detector that actually works," and backs that up by saying its detector boasts over 99.9% accuracy and a near zero false positive rate. Universities and publishers use it, as does Substack and Google Classroom. The Atlantic has reported a horror novel was pulled after a flag from Pangram, news articles have been flagged after publication, and award-winning short stories have been called into question. We wondered if its reputation held up under scrutiny.
It’s possible for an AI detector to have an excellent statistical track record but still make high-stakes calls incorrectly. When it does, the person on the receiving end isn’t particularly interested in the confidence interval. To find out, we tested Pangram’s claims, looked at its published datasets, reviewed some external studies, and spoke to users. None of this really surprises us. Back in 2024, StealthGPT's founder predicted that as AI-generated text became more prevalent, AI detection would become just as widespread — and that race is exactly what's playing out with Pangram today.
What Is Pangram AI Detector?
Pangram has frequently been in recent news over AI detection, and rightfully so. Founded in 2023 by former Tesla and Google researchers, the Brooklyn-based startup offers AI text and image detection tools to publications and organizations who need to know if a story is written by a human or a machine.
Pangram uses a machine learning approach that differs from the perplexity and burstiness metrics touted by other AI detectors: It uses millions of documents to build a classifier that looks at both human and machine-generated text, so the company’s model can pick out differences between writing by a human or an AI. Pangram points out the Declaration of Independence would be incorrectly identified as AI-generated by such tools because of its low perplexity score, which means it is easy to predict the next word. That’s not supposed to happen with simple human writing.
How Does Pangram AI Detector Work?
Pangram 4 returns a multi-part classification with both segment-level labels (Human, AI-Assisted, or AI-Generated) and document-level proportions of Human, AI-Assisted and AI-Generated content, as well as a headline Human, Mixed or AI classification. The Pangram 4 model is built upon a sparse mixture-of-experts language model and is trained in two stages with a dataset of licensed human writing and in-house machine-generated text.
The data has multiple classification heads including:
Segment-level head: the aggregate percentage of machine involvement
Token-level head: per-token Human, AI-Assisted or AI-Generated labelings
Mixed-authorship head: identifying texts with multiple types of authorship
Humanizer head: trained separately for detecting AI text that has been run through "humanizing" tools.
We refer to this as the Pangram 4 model. Pangram does not use text from users or from customer API calls for training.
A technique is used called "synthetic mirroring", in which AI is asked to mimic human-written text to produce in-house machine-generated text. This ensures that Pangram’s classifier is able to learn human versus AI text based on stylistic, rather than topical, differences.
How Accurate Is Pangram? (Our Test Results)
The following benchmark was done by StealthGPT on September 6th, 2026, to determine the performance of our new Super model. We took a sample of 100 AI-written texts, processed them using our Super model only once, and tested the result on Pangram (V4), GPTZero, Winston, and Originality.ai's Turbo model. For clarity, every test was done in each detector using default settings.
We define a Pangram or GPTZero test as a "bypass" when the result is human. For Winston and Originality, we consider a bypass as when the AI % result is lower than 50%. Please have in mind that AI-detectors all work different, and a number does not always have the same meaning. For example, a 20% result on Pangram does not equal a 20% on GPTZero. Therefore, the bypass rates have to be seen as compared to the results of that AI-detector.
For the results to be as realistic as possible, no retries were made after a failed test.
It’s important to remember that Super, tested against all four detectors, was built specifically to defeat Pangram. And while Pangram still fared better against Super than any other detector, its claimed efficacy was nowhere to be found.
With the launch of Pangram V4, the company released a benchmark claiming their detector has a 98.83% detection rate. The benchmark claims to test against 13 commercial humanizing tools on an independent dataset. Pangram is widely regarded as the hardest detector to fool.
Super achieved a bypass rate Pangram V4 of 89% with its catch rate is 9.62%. This means that it’s failing to detect 89% of text samples that Super rewrites. And not only was Pangram missing almost all samples rewritten by Super, but those misses weren’t close calls; they were resoundingly human results. When we tested against Pangram using Super, we had 9 AI results, 6 mixed, and 285 human results. Pangram claims to be catching virtually all AI-isms in texts rewritten by humanizers. When we tested its claims specifically against Super, it was barely catching 1 in 20.
Pangram's Own False Positive Rate Explained
The latest iteration of its tool, Pangram 4, comes with a “model card” that lists a false positive rate of 0.0041 percent, or about 1 in 24,000.
That’s different from the figure the company had publicized until now, and which Pangram’s CEO Max Spero told journalists as Substack started rolling the tool out: 1 in 10,000.
Pangram maintains that the 1-in-10,000 figure is older, but is from testing on “an aggregate of public datasets containing tens of millions of written documents.” The dataset used for the model card was different, namely 1,000,000 samples of human-written English text from the FineWeb dataset.
So the false positive rate isn’t necessarily wrong, just tested on different datasets, but a visitor to Pangram’s site who is looking for an answer to the question “how accurate is Pangram” will encounter both figures. On the FAQ on its homepage, the company says 1 in 10,000, while the model card on its site reports 1 in 24,000.
Where Pangram's False Positive Rate Is Higher
The headline number hides real variance. Pangram's own model card breaks out false positive rates by domain and by language, and the spread is wide:
It is important to highlight that the confidence score is not an indication of a probability of being correct, but rather of how "peaked" the models decision is.
Pangram's documentation is surprisingly open about its limitations and the risks associated with the tool, especially in comparison to competitors. In its model card it warns:
In the FineWeb2 dataset false positives ranged from 0.0000% for Arabic, Chinese, and Japanese to 0.0361% for Ukrainian (roughly a 9x difference). False negatives ranged from 0.28% for Czech to 5.32% for Urdu (almost a 19x difference).
Pangram 4 is optimized for natural-language prose, and works best on documents with at least 50 words.
Pangram indicates that there are types of documents for which it is out of primary scope and likely to return an incorrect result:
Answers to questions that can be answered with a single fact
Very short conversational exchanges
Pangram also notes types of documents that it does cover, but in which it is more likely to return an incorrect result:
Documents that are created from templates or that are auto-generated
Instructions or technical documentation (such as a troubleshooting guide)
Documents containing lots of math or other technical content
Tables of contents or references
The documentation also includes a section on how to best prepare content for scanning. For instance, it recommends removing headers, footers, and formatting:
It also recommends uploading plain text or .docx files and avoiding PDFs because the parsing process can introduce unintended artifacts.
In its Ethics & Safety section, Pangram acknowledges that being falsely accused of submitting AI-written content can have reputational and emotional consequences and that although it takes these false positive reports extremely seriously, it recognizes that there is a non-zero error rate in its system.
Is Pangram a Good AI Detector? What Independent Studies Say
AI detectors are biased against non-native writers, which means the false positive rate is high. This is the percentage of the time a detector incorrectly labels an essay as “AI-generated” that was written by a human.
Pangram has been referenced a few times as the more accurate AI detector in cases of non-native English writers. For instance, they include the University of Chicago research which finds that Pangram has a lower false positive rate compared to other AI detection tools (GPTZero, Originality.ai, RoBERTa-based detectors) on product reviews, news articles, and résumés. In the University of Maryland research on AI detection models, Pangram outperformed the other detection tool tested. Their model card also cites Liang et al. (2023) as a benchmark, reporting a 0.000% false positive rate on the same 91-essay set. Liang et al. (2023), a research study published in a journal called Patterns, tests seven widely-used AI detectors on a benchmark of 91 TOEFL essays written by Chinese students. All seven detectors showed a false positive rate averaging 61.3%, flagging more than half of the human-written essays as “AI-generated.”
While Pangram has not been found to be as biased as some AI detectors and performs the best in the study, it can be argued that all the AI detectors are flawed in a context of non-native English writers. In a 2026 journal article from ASEAN Journal of Open and Distance Learning, AI detectors (two tools) show significant bias, in which there were opposing results from two detectors on whether ten essays written by Filipino undergraduates were human or AI-written. A research article from Ahmad Pratama at Stony Brook University examines the bias of AI detectors in flagging non-native students’ academic writing and finds that the characteristics of non-native academic writing cause high false positives, whether or not the student used AI.
What Real Users Are Saying: Pangram False Positives on Reddit and Trustpilot
AI writing detectors falsely flagging human-written content is a major issue with serious consequences. The Atlantic has published a list of such cases including a horror novel that was cancelled days before its release in the U.S., published articles in newspapers, prize-winning short stories, University inquiries, and even papers. In each case, despite Pangram's claimed false-positive odds of roughly 1-in-10,000 to 1-in-24,000, these rare errors still happened to specific individuals with serious consequences.
One example is a reviewer who was required to run her work through an AI detector because she was a Capella University student. She was shocked when she received two very different results depending on how she presented her work:
When I first ran the document, I submitted the paragraphs individually, which came back “100% human written.”
Later, I ran all paragraphs at once, combined into one cohesive document, and it came back “100% AI written.”
Other users of Pangram report having to dumb down their writing to avoid being flagged:
I have to be careful with fancy dancy words or you get caught with a false positive.
Pangram has been used by large platforms, including when Substack rolled out platform-wide AI detection using Pangram in July 2026 to writers’ dismay. Critics have also pointed to its track record of unfairly flagging writing by non-native English speakers and neurodivergent people.
There are other cases that demonstrate the unreliability of AI detectors in general. One LessWrong poster found Pangram to give very different results based on the input size:
A full document scored as AI-influenced, but a 338-word passage pulled from the middle of that same document came back as 100% human.
Overall, Pangram gets a pretty poor rating among users, averaging between 2.4 and 2.5 out of 5 stars.
Can You Beat Pangram?
AI detection isn't foolproof. For example, Pangram has been integrated into Substack, and the AI detection company has already been fooled at least once. When the integration began in July 2026, a Substack creator spent roughly 3 hours and $34 worth of Claude Code credits trying to "de-AI" AI text to outsmart Pangram. After a long and arduous struggle, they finally managed it, and then discovered that all they needed to do was replace an em dash with a colon. The text went from 100% AI-generated to 100% "human," all from changing a single character.
This could be seen as a false negative, which is problematic for AI detectors when a writer wants to pass off AI text as being human-written. Pangram has published research on what "tells" it uses, and one of its most heavily-weighted is the em dash. The company reports that they appear about 10 times more frequently in AI-generated text, but also noted that the frequency varies by model family. OpenAI models are especially heavy in em dashes, while Google's Gemini 3 Pro uses them less often than people do. Still, this example highlights the pitfalls of style-based AI detectors, as over-relying on stylistic tics can make it easier to bypass.
Pangram may also rely too heavily on its assumption of what AI text is like, as another instance suggests. In a LessWrong investigation, a user prompted a language model with a lot of context for writing a recommendation letter, and found that Pangram rated the initial draft as 100% human-written without any edits at all. The user concludes that the text is dense with detail, so Pangram's classifier expects that to mean it's human-written, likely assuming that AI text will be more generic.
That said, Pangram isn't easily fooled in a casual setting, as these instances required some amount of effort to make them occur. But it does raise some questions, as AI detectors could be robust in one area, but still able to be bypassed using techniques that are at least somewhat documented.
Pangram vs. GPTZero vs. Turnitin vs. Copyleaks
Pricing is also worth a mention if you're comparing tools directly, not just accuracy. As of mid-2026, Pangram's paid plans start around $20/month for an Individual plan (roughly $15/month billed annually), scaling to $65/month for Professional, notably higher than most competitors' entry tiers. Pangram bills per 100-word unit as of the Pangram 4 pricing update, a change from the older 1,000-word-per-scan model. A free tier is available with a limited number of daily scans.
Is Pangram AI Detector Free?
Yes, with limits. Pangram's free plan doesn't require a credit card and includes a capped number of scans per day (reported figures vary slightly by source, generally in the range of a few scans daily, translating to roughly 2,000 to 4,000 words per day), AI detection with interpretability features, and support for 20+ languages. Higher volume, plagiarism detection, and full API access require a paid plan.
Verdict: Should You Trust Pangram?
In the spirit of full disclosure, there are people getting caught by Pangram false positives, as you can read about in this Atlantic article on wrongly removed books, in media reports about articles being marked as AI-generated, 3rd party review sites, and in discussions on Reddit, for example. That's to say that while independent research shows Pangram to be better at the task at hand (in specific test cases) compared to first-generation AI detectors, the performance of its model can vary with language and domain and with how the input is chunked.
While Pangram claims to be "99.9% accurate", a 1 in 10,000 chance of a false positive could mean a significant number of people when it comes to a high-volume platform like Substack. So, do take AI detection results as a starting point for a conversation, not the ultimate ground truth. Especially if you write in English as a non-native language, if your input is a short piece of text, or your text deviates significantly from the typical long-form writing genre.
To Pangram’s credit, it is more transparent about its model weaknesses than most others but its candor is its downfall. StealthGPT’s AI humanizer is here to help those who want to keep their content undetected and defend against false positives.
FAQ
Is Pangram accurate?
Pangram's published false positive rate ranges from 0.0041% (about 1 in 24,000, per the Pangram 4 model card) to "1 in 10,000" (the figure used in company marketing and press interviews). Independent research generally ranks it ahead of competing detectors, though accuracy varies significantly by writing domain and language.
What is Pangram's real false positive rate?
It depends on the text type and language. Pangram's own data shows false positive rates ranging from 0.000% on creative writing and speeches up to 0.078% on poetry, and from 0.0000% in several languages up to 0.036% in Ukrainian.
Can Pangram be wrong?
Yes. Pangram's model card explicitly acknowledges a non-zero error rate, and real-world reports, from Trustpilot reviews to Atlantic-covered publishing incidents, document specific false positive cases.
Does Pangram discriminate against non-native English speakers?
Independent research on AI detectors broadly has repeatedly found elevated false positive rates for non-native English writing. Pangram's own model card reports a 0% false positive rate on the specific non-native English benchmark it cites (Liang et al., 91 essays), which is better than the average finding for detectors as a category, but the pattern is well-documented enough across the field that it's worth factoring in regardless of vendor.
Can an AI API beat Pangram?
Yes, specifically StealthGPT’s Super can beat Pangram AI. As of its release on September 7, 2026, Super is the only AI that can beat Pagram. This goes for StealthGPT’s AI Humanizer API as well. API clients also have access to using Super in their workflows.
What text should you never run through Pangram expecting a reliable result?
Per Pangram's own stated limitations: anything under 50 words, short conversational replies, single-fact answers, source code, tables of contents, templated or automated writing, technical manuals, and math-heavy text. PDFs are also more error-prone than raw text or .docx files due to parsing artifacts.
