This classifier detects whether a code comment is written by a human or a robot. The predicted percentage is the probability of the verdict being correct. On average, 77 % of the classifier's verdicts are correct.
The classifier runs fully locally in the browser, and nothing you paste in the textbox leaves your machine.
To classify a single comment, paste it into the textbox and choose language "None". To extract all comments from a source file, choose the appropriate language.
When multiple comments are extracted from a source file, each is given a circled overall human/robot prediction percentage. Clicking the circle brings up a view for that comment specifically, including predictions for specific LLM models.
The comments are rendered with a heatmap of feature activations. Yellowish hues indicate the parts of the comment that lean toward human, and blueish the parts that lean toward robot. Any character in this heatmap can be clicked to get a view of which features activated most strongly, and which way they nudge the verdict.
The percentages are predictive probabilities, not fractions. The verdict "robot, 63 %" means there's a 63 % probability the verdict is correct, not that two-thirds of the text is AI-generated.
The percentages are based on an assumption of ignorance, i.e. that human-written and AI-generated comments are equally likely at the outset. If you want to change this assumption, toggle "Adjust prior". (Regardless of the prior, hovering the percentages will open a tooltip with the log-odds differences, which is the strength of the evidence added to the prior to produce the prediction.)
The accuracy of the LLM model breakdown is 56 %. This means that only about half the time is the LLM model judged as most likely by the classifier actually the one that generated the comments. If you are worried about misassignment, you should only believe the top model if the classifier assigns it a confidence higher than 25–30 %, depending on how sensitive you are to false positives. (10 % FPR and 5 % FPR respectively.)
The classifier is trained to recognise AI-generated source code comments. It does work for other types of AI-generated text too, but its accuracy is lower and its calibration is invalid outside of the domain it was trained on.