I don't want to lead you in the wrong direction, so I need you to clarify your situation a bit.  To be clear, are using Cain's man-in-the-middle capabilities to sniff traffic on a switched network, or are you just sniffing a non-switched network?  If you are just plan sniffing then there shouldn't be any reason that another system could see you.  Those packets are going to every system on the wire.  If you are sniffing on a switched network then there is a chance that someone could pick you out.  In fact, some IDS systems do this automagically.  If the ARP table is examined it would reveal that more than one IP address is related to the same MAC address, which will start raising flags.  From the aspect of a windows system, it will check the local ARP tables after certain events, and as part of that check it looks for IP conflicts.  It wouldn't know that you stole the IP for malicious reasons, it would just know that someone else is claiming rights to the IP.