I've only had ONE run-in with a virus, and that was years ago when I stupidly used some school computer to work on a little qbasic project and got my floppy disk infected with Empire.Monkey.B (and, subsequently, my computer.)

I download things off the internet from trusted sites, I keep my virus defs up to date and occasionally scan my system, but don't leave any active scanning crap running. I have no firewall, and I never have any problems.

Just because soemone portscans your system LOOKING for trojans, doesn't mean you're infected. They are either 1) scanning entire IP ranges which you happen to be a part, or 2) using a modified gnutella client to harvest IP's to target.