The <saml:Audience> element is used to compensate for SAML products that produce unusual assertions that otherwise can't be processed. This is more likely when using SAML 1.1 than with SAML 2.0.
If an SP refuses to accept an assertion due to an unacceptable audience condition, adding appropriate <saml:Audience> elements MAY correct the problem, but this is rarely the right thing to do. Usually it's a sign of deeper problems that need to be corrected in order for the system to work properly anyway.
In particular, NEVER use this element with a Shibboleth IdP. The only situation in which it would change the software's behavior is one in which the IdP has failed to properly recognize the SP anyway. Instead of using this element, correct the Metadata that is causing the problem.