.ashx context.response.writefile - Use file on remote server?

I'm using an .ashx handler to retrieve an image file...  It queries a local database, which retrieves the URL of an image which is hosted on Amazon S3.
But I can't use context.response.write(AmazonS3URL) because it returns the error '...is not a virtual path'. 
Can context.response.write only return files that are on the local machine?  Or is there another function to call remote files?
...