Sometimes my penis doesn't stay hard in the middle of intercourse. maybe the condom?

You simply suffer from erectile dysfunction, it happens to lots of guys. There are several ways to get over this. There are medications you can get, exercises which you can perform, and special condoms you can buy which will help you to stay harder longer. I suggest you consult your doctor as to what's best for you.