Can someone tell me the name of this book?

I read this book as a child and I can't find it anywhere.
I remember the plot; here goes nothing: It was about these magical beings (Witch with black hair and wizard?) who come to earth from a different world to find their prince who was taken/hidden at birth. They think they find him living with a wealthy woman, but it turns out to be the boy's servant (Ben?). So they take him back to his parents, happily ever after. I faintly remember it taking place in England, maybe.
Help, please? I really want to read this again.