No guarantees but that's the 1st place to look. Both for yourself & the other party.

I guess it's in some ways similar to browsing, one moment you can direct connect to them, a few mins later you can't or vice versa. Though if you can once, then generally you can continue to do unless their bandwidth & open slots is obviously filled up. I've been able to browse people I one moment could DC to & some time later couldn't. But browsing is similar in the respect, one moment you can browse them & perhaps the next you can't or your browse finds less files in the browse. So all these issues are affected by both each user's upld & downld capabilities at that particular moment as well as traffic & peer connections in-between.