What sentiment analysis is, and how word lists differ from machine learning

Sentiment analysis is the job of reading a piece of text and deciding whether the writer sounds positive, negative or neutral about something. For posts on X, that might mean sorting 300 replies to a product announcement into "people like it", "people are angry" and "people are asking questions". A person can do this by reading. Software does it faster and less carefully, and how much less carefully depends on the method.

There are two broad families of methods. One counts words from a list. The other learns patterns from examples. Most real tools mix them, but the difference explains almost every surprising result you will see.

Lexicon methods: a list of words with weights

A lexicon is a dictionary where each entry carries a sentiment value. "Excellent" might be +2.8, "meh" −0.5, "scam" −2.5. To score a post, the software finds the words that appear in the list and adds their values. A positive total means a positive post.

Plain addition breaks quickly, so lexicon tools add rules:

• "Not good" should not count as good, so a negation rule looks back a few words for "not", "never" or "isn't" and flips the value.
• "Really bad" is worse than "bad", and "slightly annoying" is milder than "annoying". Intensifiers and softeners scale the word that follows.
• In "slow delivery but great quality", most readers weigh the part after "but" more, and a contrast rule does the same.
• On social platforms, 😡 or 🙌 often carries more meaning than the words around it, and three exclamation marks usually add force.

Academic word lists have been around for years. Bing Liu's group at the University of Illinois Chicago published an opinion lexicon of several thousand English words built from customer reviews. VADER, released by C.J. Hutto and Eric Gilbert in 2014, was tuned for social media text and handles capitals, emoji and slang better than older lists. The checker on this site uses a smaller list written for Twitterment, with the same kinds of rules.

What lexicons are good at

The main strength is transparency. When a lexicon tool labels a post negative, you can see exactly which words did it. If "sick" is being counted as negative in a skateboarding community where it means great, you can spot the mistake and adjust for it.

Lexicons also need no training data, run instantly, and behave the same way every time. That makes them a reasonable first pass when you want a rough split of a few hundred posts and plan to read the interesting ones yourself.

Where they fail

A word list has no idea what the post is about. "This movie was so bad it was brilliant" contains both "bad" and "brilliant", and the score will depend on which weighs more in the list. Sarcasm is worse: "Great, another update that logs me out" is a complaint written with a positive word. Replies are hardest of all, because "same" or "finally" only make sense next to the post they answer.

Coverage is the other problem. A list knows the words someone put in it. New slang, brand names, misspellings and community jargon fall through as neutral.

Machine-learning methods: learning from labeled examples

The second family starts with examples rather than rules. Someone collects thousands of texts that people have already labeled positive or negative, and a model learns which features predict each label. Bo Pang, Lillian Lee and Shivakumar Vaithyanathan showed in 2002 that standard classifiers trained on movie reviews beat simple word counting, and also found that sentiment was harder to classify than topic.

Early models still worked mostly with individual words and short word pairs, but they learned the weights from data instead of from a person's judgment. Modern models built on large pretrained language models go further. They read the whole sentence, and often the surrounding text, so they can sometimes tell that "so bad it was brilliant" is praise.

What machine learning is good at

Trained models usually beat word lists on accuracy when they are tested on the same kind of text they learned from. They handle word order, odd phrasing and some sarcasm, and a model fine-tuned on social posts will pick up common slang automatically.

Where it fails

The accuracy comes with costs. A model is only as good as its training labels, and those labels come from people who disagree with each other more often than you might expect. Ask three colleagues to label the same 100 replies and they will split on a fair number of them, mostly the neutral and mixed ones.

A model trained on product reviews can stumble on political arguments, sports banter or gaming communities. It can also be hard to say why it made a call, which matters when someone asks you to defend a number in a meeting. Large models are also slower and usually run on a server, which means sending your data somewhere.

Which one should you use?

For a small brand or a one-off check, a lexicon tool is often enough, as long as you treat it as a sorting aid. Paste the replies, look at the split, then read the lines in each group and the ones flagged as possibly sarcastic. If the tool says 70 percent positive and your own reading of 30 random lines agrees, you have a usable answer.

Machine learning earns its cost when you have thousands of posts a day, need a trend line over months, or work in a domain where word lists keep getting it wrong. Even then, it helps to hand-label a few hundred posts from your own data and check the model against them before trusting it.

Some questions are not worth automating at all. If you need to know why people are upset, sentiment scores won't tell you. The reasons are in the text, and reading it is still the fastest way to find them.

A quick test you can run

Take 20 posts you have already read and labeled yourself. Paste them into the Twitterment checker and compare its labels with yours. Count how many it got right and look closely at the misses. Most will fall into a few groups: sarcasm, replies that need context, and words used in an unusual sense. Once you know which mistakes the method makes on your kind of text, you know how far to trust it on the next batch.

Questions

Is sentiment analysis the same as emotion detection?

Not quite. Sentiment analysis usually sorts text into positive, negative and neutral, sometimes with a strength score. Emotion detection tries to name the feeling, such as anger, joy, fear or surprise. Two angry posts and two sad posts can all be negative while carrying very different emotions.

Which method does Twitterment use?

A word list with rules. Every score on the checker can be traced back to the words that produced it, which makes it easy to check and easy to disagree with. It is fast and runs offline in your browser, and it shares the usual weaknesses of lexicon methods with sarcasm and context.

Twitterment is independent and not affiliated with X Corp. Features and search operators on X change from time to time; if something here doesn't match what you see, check X's own help pages.

Open the sentiment checker