One of the most common situations where atheists feel awkward is a gathering where someone initiates a public prayer, and asks people to join in (usually silently, with heads bowed and/or hands folded). Here’s my take on why, in most cases, the polite thing to do is to bow your head and play along:
Taking part in a ritual doesn’t imply buying into all the cultural/social/religious baggage associated with it. There are lots and lots of cases where people engage in gestures or traditions that carry meanings they don’t remotely intend.
We shake hands with people in lots of situations where neither of us are armed and there’s no need to demonstrate that our sword hand is empty. Some of us knock on wood (or say “knock on wood”) even if we’re not pagans who believe trees hold spirits who can watch over our needs. And likewise, bowing your head while someone utters a memorized, ritualistic speech doesn’t mean you wholeheartedly endorse the underlying concepts.
You could argue, of course, that it’s rude of the person initiating the prayer to do so in a situation where there are (or may be) atheists or other non-believers present. That may well be true (although there are situations where prayer is an accepted part of the deal, like a Christian wedding — see below).
But one of the big rules of etiquette is that it’s rude to point out that someone else is being rude. At least, it’s rude to point it out directly. There are some passive-aggressive moves that some etiquette experts recommend for people who simply must say something, but if you’re trying to maintain good social relationships (which is, after all, what etiquette is for), you don’t help to accomplish that by pointing out that someone else is being an idiot or an asshole.
And you could also argue that silently going along with a prayer makes it harder for atheists to stand up and be counted, and creates a deceptive picture of just how few atheists there are out there. If you think there’s a moral imperative that’s more important than being polite, then do what you think is right — but don’t be surprised if people think you’re rude. Because basically, you are. That’s not necessarily a bad thing — as someone said, polite people rarely make history. But it’s still a thing.
And if you’re at a Christian wedding, and the minister calls for a moment of silent prayer, it’s perfectly appropriate for you to bow your head. Just as it’s perfectly appropriate for a Christian at a Buddhist wedding to bow her head, or stand, or kneel, if that’s what everyone else is doing. A wedding is about the happy couple, not about you. And if you didn’t want to be exposed to Christian religion, the time to make that decision was before you walked through the Christian church doors and took your seat at the Christian church pew.
There are some other prayer-related etiquette issues (What about holding hands to say grace? What if someone asks you to offer a prayer? What if a prayer you’re going along with turns nasty?), but I think this will do for now.