Actually, I know some excellent graphic designers who are mildly to severely color blind. They all do AMAZING work with black and white. They all have highly sophisticated understandings of shape, line, form, white space, contrast, etc. I say go for it! Emphasize your strengths.