"I'd get terrible headaches if I stopped"
You may have tried cutting down before, and you remember what happened. Within a day you had a headache like a band round your skull, you were foggy and flat and short-tempered, and by day three you'd concluded that your body needed coffee and gone back to it, feeling better within the hour.
Laura Juliano and Roland Griffiths reviewed sixty-six studies of caffeine withdrawal in 2004, and the picture they drew is worth having in full, because it's both worse and better than you think. Ten symptoms were well enough supported to count as real. Headache, fatigue, decreased energy, decreased alertness, drowsiness, decreased contentedness, depressed mood, difficulty concentrating, irritability, and feeling foggy. Flu-like symptoms, nausea and muscle stiffness were also likely. In the experimental studies, about half of people got a headache, and about one in eight had distress or impairment bad enough to interfere with their day. Symptoms typically began twelve to twenty-four hours after the last dose, peaked at twenty to fifty-one hours, and lasted between two and nine days. Going without doses as low as a hundred milligrams a day, which is one mug of instant, was enough to produce symptoms.
So you're right. If you stop, you'll probably feel worse for a few days, and there's a fair chance of a headache. That's not weakness and it's not imagination. The reviewers were clear that expectation isn't what drives it.
Now the other half. Two to nine days. Not weeks, not permanently. Your body doesn't need coffee. It has adapted to it and needs a few days to adapt back, and then the shortfall is gone, for good, and you wake at the level. What you experienced on day three of your last attempt wasn't proof of a permanent need. It was the peak of a short process that you stopped just before it ended.
The reviewers also say something that I think is the centre of this book. Avoiding withdrawal, they found, plays a central role in habitual caffeine consumption. People keep drinking coffee in large part to not feel what stopping feels like. The morning cup is taken at the moment the overnight withdrawal begins to bite. The afternoon cup is taken as the morning's dose wears off. The habit is arranged, without anyone deciding it, to keep the shortfall from ever being felt in full, and the price of that arrangement is the night.
There's also no rule that says you have to stop all at once. Chapter 18 describes bringing it down over two or three weeks, so that the shortfall never gets deep. Most of the headache is avoidable that way. And the destination isn't necessarily zero. It's whatever amount you choose to drink because you like it, taken early enough not to cost you the night.