Having been in close contact with tip workers, but not actually a tip worker (hotel manager — aaages ago). I’d have handed you at least a 20 before saying “Y’know what? keep the change.”
I never realized that I was a good tipper until my wife and I moved to a new town. We went to each place we were ‘regulars’ at just to say “goodbye, You won’t see much of us going forward; it’s not your fault, but we’re 20 miles away now…”. and a couple of waitstaff in some of those places mentioned that they’d miss the tips.
Here’s the thing: if you are a regular, it really pays to be an at-least-decent tipper. There was one place we went to regularly that had a dish I loved — but it took 25–30 minutes to cook. If we were waiting for a table, one of the staff would find us and ask if we wanted that dish (yet again), and, if we did, they’d put an order in. Up to those goodbye visits, I thought they were asking so they could get another cover on a table
BTW, if that guy calls again with an after hours request, maybe tell him that there are “after hours prices” and charge him, say, $40 — and he can keep his change.