I of course mean portforwarding past the gateway. I use
rdr le1 0/32 port 6364 -> 192.168.0.1000 port 6364

But I still have problems connecting 80% of the times, but no push requests at least.