It sounds like you just have to provide "clear directions" on where to find the source code, not host it on your servers. And that's only GPLv3.
That's not correct. You have to distribute the source yourself according to the Free Software Foundation. Specifically:
I downloaded just the binary from the net. If I distribute copies, do I have to get the source and distribute that too?
Yes. The general rule is, if you distribute binaries, you must distribute the complete corresponding source code too. The exception for the case where you received a written offer for source code is quite limited.
There are really two reasons for this. The first is availability. So that when you distribute the binaries, you are distributing the source, and will continue to do so. That way, if the original source goes dark (offline, out of business, etc), you are still distributing the binaries as well as the source to those who got the binaries from you, so those recipients can continue to get the source even though the original provider has disappeared.
The second is fairness. This way, if a repackaged, modified or redistributed version of a given program were to become wildly popular, the recipients of that program aren't all going back to the original provider (who may be comparatively tiny and have far less resources) and sucking up their bandwidth (and, thus, money) or using their resources to mail out physical copies. This keeps people from leeching off another open source provider's resources.