Do I pick the guy i'm dating or stick to my brothers over protective ways?

I've been seeing this guy for about a year now (casual, monogamous relationship, agreed by both of us) and we've finally had "the talk". He tells me how he feels about me etc. however, the reason he's afraid to fully commit to me Is because of my brother (let me explain).My brother once kicked him out of my house when he found out he had spent the night with me (bearing in mind they had met once before and they seemed to get on pretty well).
Now I'm in a predicament because I don't know whether I should just confront my brother about this and just make him accept that this is who I choose to date and he should just mind his own business OR just stop seeing this guy altogether - because I'm afraid of my brothers over protective ways?
I'm a twenty four year old woman, who works and lives on her own. Me and my family are very close but my brother never approves of the relationships that I have been in. PLEASE HELP with constructive advice..! Thank you in advance!
Asked Dec 29, 2016

TIP: If it's not your answer to this question, please click "Leave a Comment" button under the question to communicate with the question owner.