factor analysis with dummy variables

I'm trying to figure out if it makes sense to use factor analysis when
dealing exclusively with dummy variables. I'm worried that since I'll
be randomly assigning numeric values to such parameters as gender,
race, etc any sort of a correlation I find behind the varialbes will
be pretty meaningless. Am I correct in this logic?

.