Axioms are still statements in a language. The universe is not a language.
Think of this way. The completeness theorem provides a way of verifying the axioms in terms of mathematical and physical syntax in terms of model theory. There are many physical models eg: Standard model, Relativity, String theory etc. So while the universe is not a language, it can however be described using axioms in terms of the models mentioned above.
Axioms can be postulated and rejected arbitrarily. This is not true of universe.
Axioms can be postulated and rejected arbitrarily, but there is a limit to this assumption and deduction. This limit is set by the observation model itself, the universe. The universe forces scientists to confirm to a set of standard axioms as accepted under the above set of theories. The problem so far is that certain axioms in certain models are not consistent with the other models. We can understand this in terms of discrepancies between our models and the physical world. So while Axioms can be postulated and rejected arbitrarily only upto the extent of the model system chosen. Our current theories in physics require a standard sets of axioms expressed by their own syntax i.e maths, physics etc.
I repeat: axioms are statements in a language. The universe is not an axiom, nor a set of axioms.
The critical aspect you need to comprehend is that in first degree logic, axioms assumed need not be proven by itself. Since first degree logic refers to the standard syntax of mathematics and physics, all our attempts are in the direction of reducing the universe into self evident blocks of axioms into one standard model of this universe.
Consider this when a kid or a nobel prize winner says 2 + 2 = 4, he does not need to prove the rules of mathematics or the number 2 itself or its additive number 2. He begins on a base of syntax of an approximate physical model and then uses that syntax to construct another axiom. This universe is a set of axioms confirming to a physical model, we just need to find the right one yet.
Non-sequitur. Nothing is ever "proved of the universe." The universe simply is.
When you say 'the universe simply is', it is assumption resting on a self evident fact (an axiom) i.e the validity of the existence of the universe. When you 2 + 2 = 4, you are assuming that everything (axioms) leading to maths model is undeniable by itself and you are using the language of this maths model to derive a further truth.