A bastion host is sort of a front line machine offering a service of some kind such as email on the outside of a network firewall. It is only allowed to connect only on a secure channel to an internal server behind that firewall where the actual important data (email database) is stored.
The idea is that you minimize your external exposure of a sensitive server. It will be harder to hack the external one as long as you only run the single service on it. And, theoretically the internal server will be harder to compromise since the only rule through the firewall allows incoming traffic from that bastion host.
So in your case the firewall is not a bastion host but it's the more important piece of one.
Gaidin