Security through obscurity is not typically recommended.

If the files are there, Internet-accessible, there will always be potential for someone to access them.

If you only want Server A and Server B to share the files, I would suggest you look into some sort of PKI implementation to encrypt the data that's shared so that only those servers can access it.

BillV

I don't see why someone wouldn't be able to access the file on your server.   You didn't mention anything that would prevent that.   

At the very least you could consider using basic HTTP authentication. This would require setting up a .htaccess and .htpasswd file on the webserver (assuming you're using apache).

http://httpd.apache.org/docs/2.0/howto/auth.html

It's worth having a look at basic auth since it's fairly easy to get the hang of and implement.

This comes with the caveat that it's not a robust security mechanism but it's much better than using 'secret' URLs. They can too easily fall prey to insecure anonymous FTP browsing (I've seen this on some ISPs), mod_speling (http://httpd.apache.org/docs/2.0/mod/mod_speling.html) and other common features and pitfalls.

Jimbob

One thing to remember is that the referrer header is sent by the client i.e. the web browser. It therefore cannot be trusted as a security token. That said it is an additional barrier and I'm all for defense in depth 

Jimbob