How choose between 32-bit and 64-bit?

I see sql express limits says it can only use up to 1 gb memory, and I assume that apply to both 32-bit and 64-bit. So what other things can make sql express 64-bit better than 32-bit? And I have impression that 32-bit can have smaller overhead than 64-bit.
...