Text Sentiment Analyzer

Lexicon-based sentiment scoring assigns each word a valence, here on a scale from minus 3 to plus 3, then applies contextual rules before summing. Two rules do most of the work. Negation is detected by scanning the three tokens preceding a sentiment word for markers such as not, never, no or a contracted form; a match flips the sign and scales the magnitude by 0.75, because not good is milder than terrible. Intensifiers immediately before a sentiment word multiply it by factors ranging from 0.5 for downtoners like slightly to 2.0 for amplifiers like extremely. The total is normalised by the number of sentiment-bearing words times the maximum valence of 3 and clamped to the range minus 1 to plus 1. The reported bands put anything within 0.05 of zero as neutral, 0.2 as positive or negative, and 0.5 or beyond as strongly so. Subjectivity is the share of all tokens that carry any valence; factual reporting typically stays under about 10 percent while opinion writing and product reviews run well above 25. Intensity is the mean absolute valence, which separates mild approval from enthusiasm. The known failure modes are inherent to the approach. Sarcasm inverts meaning without changing vocabulary, domain shift breaks assumptions, and negation scope is genuinely ambiguous beyond a short window. Lexicon methods typically reach 60 to 75 percent agreement with human annotators, against upwards of 90 percent for fine-tuned transformer classifiers, and short inputs of a handful of words give unstable scores.

Quantifying attitude in language predates computers. Charles Osgood, George Suci and Percy Tannenbaum's semantic differential work of 1957 established that judgments about words load heavily on an evaluative good-versus-bad dimension. The first computational implementation followed in 1966, when Philip Stone and colleagues at Harvard released the General Inquirer, a content analysis system built around hand-coded dictionaries that tagged words with categories including positive and negative affect. Modern sentiment analysis grew out of the explosion of online reviews. In 2002 Peter Turney showed that unsupervised orientation scoring could classify reviews, and Bo Pang, Lillian Lee and Shivakumar Vaithyanathan presented Thumbs up? at EMNLP the same year, framing the task as supervised classification over movie reviews and establishing the benchmark tradition. Finn Arup Nielsen released the AFINN word list in 2011, hand-scoring roughly 2,477 English terms from minus 5 to plus 5 specifically for microblog text. C. J. Hutto and Eric Gilbert published VADER at ICWSM in 2014, adding precisely the rule layer used on this page, including negation flipping, degree modifiers, and boosts for punctuation and capitalisation, with weights tuned against crowdsourced ratings. Transformer models fine-tuned on labelled corpora overtook lexicons on benchmarks after 2018, though lexicons remain in use wherever an auditable, explainable score is required.