Yet another inane amateur Godel question

From my partial understanding, the proof seems to contain in outline
the idea that any sufficiently strong formal system for generating
proofs allows formation of the sentence "this sentence is
undecidable", with the obvious result. One's first reaction is that
there should be a way to rule out such things as valid sentences:
after all, if one is out to prove things about the integers, say, such
a self referential "theorem" has nothing to say about the integers, so
who cares that it is in fact undecidable?

OK.... this is based on shoddy amateur understanding, but at least I'm
not asking if we've proved that all human knowledge must be
incomplete, or something. ;-)
.