Like tshdos mentioned, there are "host caches" that hold the last, say 20, IP addresses that connected previously. Another option is a local host cache, which simply stores all (or some) IP addresses discovered while it was in session, and uses that for the next time (so to avoid using a central host cache). And obviously, it'll also work if someone just e-mails your an IP address, or on forums like this, usenet or perhaps even IM/IRC.

-- Mike