I have a program that parser html and collecting urls. The problem is when checking for duplicates before I add them to an arraylist is that I cannot distinguish between types of the following format:
http://forums.asp.net/AddPost.aspx?ForumID=15
http://forums.asp.net/AddPost.aspx?ForumID=15/helloworld.asp
http://forums.asp.net/AddPost.aspx?ForumID=15/helloworld.asp/byeworld.html
Now if I captured the first url, how can I avoid collecting the next few since they are not technically duplicates. Any ideas please?
...
Go to the complete details ...