:orphan:

.. _pulp-broker-settings:

Pulp Broker Settings
====================

Pulp requires a message bus to run. Either Qpid or RabbitMQ can be used as that message bus. Pulp
is developed and tested against the Qpid C++ server v0.22+ and is configured to expect Qpid on
localhost without SSL or authentication by default. This documentation identifies changes necessary
for the following configurations:

   * Pulp Broker Settings Overview
   * Configure Pulp to use Qpid on a different host
   * Configure Pulp to use Qpid with SSL
   * Configure Pulp to use RabbitMQ without SSL
   * Configure Pulp to use RabbitMQ with SSL


Pulp Broker Settings Overview
-----------------------------

Pulp uses the message broker in two ways:

    * For Pulp Server <--> Pulp Consumer Agent communication such as a server initiated bind or
      update.

    * For Pulp Server <--> Pulp Worker asynchronous, server-side tasks such as syncing, publishing,
      or deletion of content.

Pulp Server settings are contained in ``/etc/pulp/server.conf`` and are located in two sections
corresponding with the two ways Pulp uses the message broker. The Pulp Server <--> Pulp Consumer
Agent communication settings are contained in the ``[messaging]`` section. The asynchronous task
settings are contained in the ``[tasks]`` section. Refer to the inline documentation of those
sections for more information on the options and their usage.

All settings in ``[tasks]`` and ``[messaging]`` have a default. If a setting is not specified
because it is either omitted or commented out, the default is used. The default values for each
option are shown but commented out in ``/etc/pulp/server.conf``.

Pulp Consumer Agent settings are contained in ``/etc/pulp/consumer/consumer.conf`` in the
``[messaging]`` section and define how the Consumer Agent connects to the broker to communicate
with the Pulp Server. The ``[messaging]`` section of ``/etc/pulp/consumer/consumer.conf`` on each
Pulp Consumer and the ``[messaging]`` section of ``/etc/pulp/server.conf`` on each Pulp Server need
to connect to the same broker for correct operation. The values and settings in
``/etc/pulp/consumer/consumer.conf`` correspond with the settings in ``/etc/pulp/server.conf``, but
uses a slightly different setting names. Refer to the inline documentation in the ``[messaging]``
section of ``/etc/pulp/consumer/consumer.conf`` for more information on how to configure the
settings of a consumer.

These two areas of Pulp can use the same message bus, or not. There is not a requirement that these
use the same broker.

To apply your changes after making any adjustment to ``/etc/pulp/server.conf``, you should restart
all Pulp services on any Pulp Server using the ``/etc/pulp/server.conf`` file edited. To apply your
changes made to a ``/etc/pulp/consumer/consumer.conf`` file, restart the Consumer Agent
(``goferd``) on any Consumer that uses that file. Normally each configuration file is kept
individually on each computer (Server or Consumer), and in those cases you only restart the
corresponding service on that specific machine. For more custom environments where config files are
shared between servers or consumers you may need to restart services on multiple computers.


Qpid on localhost (the default settings)
----------------------------------------

The default Pulp settings assume that both Pulp Server <--> Pulp Consumer Agent communication and
Pulp Server <--> Pulp Worker communication use Qpid on localhost at the default port (5672) without
SSL and without authentication. All settings in the ``[messaging]`` and ``[tasks]`` sections are
commented out by default, so the default values are used. The defaults are included in the
commented lines for clarity.
::

    [messaging]
    # url: tcp://localhost:5672
    # transport: qpid
    # auth_enabled: true
    # cacert: /etc/pki/qpid/ca/ca.crt
    # clientcert: /etc/pki/qpid/client/client.pem
    # topic_exchange: 'amq.topic'

    [tasks]
    # broker_url: qpid://guest@localhost/
    # celery_require_ssl: false
    # cacert: /etc/pki/pulp/qpid/ca.crt
    # keyfile: /etc/pki/pulp/qpid/client.crt
    # certfile: /etc/pki/pulp/qpid/client.crt
    # login_method:

The default settings of a Pulp Consumer Agent are found in ``/etc/pulp/consumer/consumer.conf`` and
assume Qpid is running on localhost at the default port (5672) without SSL and without
authentication. In almost all installations, at a minimum, the ``host`` attributed will need to be
updated. The default configuration is shown below. If ``host`` in the ``[messaging]`` section is
blank, the ``host`` attribute in the ``[server]`` section of ``/etc/pulp/consumer/consumer.conf``
is used, which defaults to ``localhost.localdomain``.
::

    [messaging]
    scheme = tcp
    host =
    port = 5672
    transport = qpid
    cacert =
    clientcert =


Qpid on a Different Host
------------------------

To use Qpid on a different host for the Pulp Server <--> Pulp Consumer Agent communication, update
the ``url`` parameter in the ``[messaging]`` section. For example, if the hostname to connect to is
``someotherhost.com`` uncomment ``url`` and set it as follows:

    ``url: tcp://someotherhost.com:5672``

The ``/etc/pulp/consumer/consumer.conf`` file on each Pulp Consumer also needs to be updated to
correspond with this change. Refer to the inline documentation in
``/etc/pulp/consumer/consumer.conf`` to set the configuration correctly.

To use Qpid on a different host for Pulp Sever <--> Pulp Worker communication, update the
``broker_url`` parameter in the ``[tasks]`` section. For example, if the hostname to connect to is
``someotherhost.com`` uncomment ``broker_url`` and set it as follows:

    ``broker_url: qpid://guest@someotherhost.com/``


.. _pulp-broker-qpid-with-username-password:

Qpid with Username and Password Authentication
----------------------------------------------

The Pulp Server <--> Pulp Consumer Agent only support certificate based authentication, however the
Pulp Server <--> Pulp Worker communication does allow for username and password based auth.

Pulp can authenticate using a username and password with Qpid using SASL. Refer to the Qpid docs
on how to configure Qpid for SASL, but here are a few helpful pointers:

1. Ensure the Qpid machine has the ``cyrus-sasl-plain`` package installed. After installing it,
   restart Qpid to ensure it has taken effect.

2. Configure the username and password in the SASL database. Refer to Qpid docs for the specifics
   of this.

3. Ensure the qpidd user has read access to the SASL database.

After configuring the broker for SASL, then configure Pulp. This section explains how to configure
Pulp to use a username and password configured in Qpid.

Assuming Qpid has the user ``foo`` and the password ``bar`` configured, enable Pulp to use them by
uncommenting the ``broker_url`` setting in ``[tasks]`` and setting it as follows:

    ``broker_url: qpid://foo:bar@localhost.com/``


Qpid on a Non-Standard Port
---------------------------

To use Qpid with a non-standard port for Pulp Server <--> Pulp Consumer Agent communication, update
the ``url`` parameter in the ``[messaging]`` section. For example, if Qpid is listening on port
``9999``, uncomment ``url`` and set it as follows:

    ``url: tcp://localhost:9999``

The ``/etc/pulp/consumer/consumer.conf`` file on each Pulp Consumer also needs to be updated to
correspond with this change. Refer to the inline documentation in
``/etc/pulp/consumer/consumer.conf`` to set the configuration correctly.

To use Qpid with a non-standard port for Pulp Sever <--> Pulp Worker communication, update the
``broker_url`` parameter in the ``[tasks]`` section. For example, if Qpid is listening on port
``9999``, uncomment ``broker_url`` and set it as follows:

    ``broker_url: qpid://guest@localhost:9999/``


Qpid with SSL
-------------

SSL communication with Qpid is supported by both the Pulp Server <--> Pulp Consumer Agent and the
Pulp Server <--> Pulp Worker components. To use Pulp with Qpid using SSL, you'll need to configure
Qpid to accept SSL configuration. That configuration can be complex, so Pulp provides its own docs
and utilities to make configuring the Qpid with SSL easier. You can find those items in the
:ref:`Qpid SSL Configuration Guide <qpid-ssl-configuration>`.

After configuring the broker with SSL and generating certificates, you should have a CA
certificate, a client certificate, and a client certificate key. SSL with Qpid is by default on
port 5671, and this example assumes that.

To configure Pulp Server <--> Pulp Consumer Agent communication to connect to Qpid using SSL, uncomment
and set the following settings in the ``[messaging]`` section. The below configuration is an
example; update ``<host>`` in the ``url`` setting and the absolute path of the ``cacert`` and
``clientcert`` settings for your environment accordingly.
::

    [messaging]
    url: ssl://<host>:5671
    cacert: /etc/pki/pulp/qpid/ca.crt
    clientcert: /etc/pki/pulp/qpid/client.crt


The Pulp Server <--> Pulp Consumer Agent SSL configuration requires the client keyfile and client
certificate to be stored in the same file.

The ``/etc/pulp/consumer/consumer.conf`` file on each Pulp Consumer also needs to be updated to
correspond with this change. Refer to the inline documentation in
``/etc/pulp/consumer/consumer.conf`` to set the configuration correctly.

To configure Pulp Server <--> Pulp Worker communication to connect to Qpid using SSL, uncomment and
set the following settings in the ``[messaging]`` section. The below configuration is an example;
update ``<host>`` in the ``broker_url`` setting and the absolute path of the ``cacert``,
``keyfile``, and ``certfile`` settings for your environment accordingly.
::

    [tasks]
    broker_url: qpid://<host>:5671/
    celery_require_ssl: true
    cacert: /etc/pki/pulp/qpid/ca.crt
    keyfile: /etc/pki/pulp/qpid/client.crt
    certfile: /etc/pki/pulp/qpid/client.crt
    # login_method:


The Pulp Server <--> Pulp Worker communication allows the client key and client certificate to be
stored in the same or different files. If the key and certificate are in the same file, set the
same absolute path for both ``keyfile`` and ``certfile``.

.. note::

     If your Qpid broker requires authentication with ``auth=yes`` and requires SSL client
     authentication with ``ssl-require-client-authentication=yes`` then you may want to have Pulp
     authenticate using the ``EXTERNAL`` method. To configure this you will need to:

          1. Set ``login_method`` to ``EXTERNAL``

          2. Ensure that the broker string contains a username that is identical to the ``CN``
             contained in the client certificate specified in the ``certfile`` setting of the
             ``[tasks]`` section.

     For example, if the ``cacert`` has ``CN=mypulpuser`` and connects to ``example.com`` on port
     5671, then ``broker_url`` should be set to:

          broker_url: qpid://mypulpuser@example.com:5671/


Using Pulp with RabbitMQ
------------------------
Pulp Server <--> Pulp Consumer Agent and Pulp Server <--> Pulp Worker communication should both
work with RabbitMQ, although it does not receive the same amount of testing by Pulp developers.

For a Pulp Server or Pulp Consumer Agent to use RabbitMQ, you'll need to install the
``python-gofer-amqp`` package on each Server or Consumer. This can be done by running:

    ``sudo yum install python-gofer-amqp``

Enable RabbitMQ support for Pulp Server <--> Pulp Consumer Agent communication by
uncommenting and updating the ``transport`` setting in ``[messaging]`` to ``rabbitmq``. Below is an
example:

    ``transport: rabbitmq``

The ``/etc/pulp/consumer/consumer.conf`` file on each Pulp Consumer also needs to be updated to
correspond with this change. Refer to the inline documentation in
``/etc/pulp/consumer/consumer.conf`` to set the configuration correctly.

Enable RabbitMQ support for Pulp Server <--> Pulp Worker communication by uncommenting and updating
the ``broker_url`` broker string to use the protocol handler ``amqp://``. Below is an example:

    ``broker_url: amqp://guest:guest@localhost//``


RabbitMQ with a Specific vhost
------------------------------

RabbitMQ supports an isolation feature called vhosts. These can be used by appending them to the
broker string after the forward slash following the hostname. The default vhost in RabbitMQ is a
forward slash, causing the broker string to sometimes be written with an additional slash. This
form is for clarity as the the default vhost is assumed if none is specified.

Pulp Server <--> Pulp Consumer Agent communication through RabbitMQ on a vhost is not supported.

To enable Pulp Server <--> Pulp Worker communication through RabbitMQ on a vhost, uncomment and
update the ``broker_url`` setting in ``[tasks]`` to include the vhost at the end. For example, if
the vhost is 'foo' with the rest of the settings as defaults, the following example will work:

    ``broker_url: amqp://guest:guest@localhost/foo``


RabbitMQ with SSL
-----------------
RabbitMQ with SSL support is configured the same as it is with Qpid with the only difference being
the adjustment to the ``transport`` setting in ``[messaging]`` and the protocol handler of
``broker_url`` in ``[tasks]``. Both of these sections are contained on the Pulp Server in
``/etc/pulp/server.conf``.

If RabbitMQ is using strict SSL client certificate checking, you will need to set ``login_method``
to ``EXTERNAL``. See :redmine:`1168` for more details.

The ``/etc/pulp/consumer/consumer.conf`` file on each Pulp Consumer also needs to be updated to
correspond with this change. Refer to the inline documentation in
``/etc/pulp/consumer/consumer.conf`` to set the configuration correctly.
