The persons whom the crime was against. If you punch someone, the crime was against that person. Ergo they are the victim. This is a simple case.
There may be more than one person who is the victim in a crime. If there is a terrorist attack and someone is killed, they are the victim. But their friends and families are saddened by the event - they are the victim also.