The <Listener>, <UnixListener> and <TCPListener> elements configure the "remoting" software that connects the in-process and out-of-process portions of the SP. For most purposes, you can ignore it as an internal detail that doesn't require your intervention.
This remoting layer is an inter-process communication mechanism, but is not optimized to be used between machines. It is accessed frequently, typically at least once per web request. The best analogy would be the AJP protocol connector used between Apache and Tomcat, but is generally even less efficient. Used over a domain socket or loopback connection, the speed is not a factor, but across a network, performance will suffer.