Getting help ============ If you have a problem or question, it is highly recommended that you execute the following steps **in order**. Many people have similar problems with configuration and initial setup of Open MPI |mdash| it is possible that your question has already been answered. #. :doc:`Validate your Open MPI installation `. This ensures that you have a nominally-correct Open MPI installation. #. `Check prior GitHub issues `_ and see if others have asked the same question and had it answered. #. `Check the mailing list archives `_ "search" features (or use Google) to check old posts and see if others have asked the same question and had it answered. #. If you do not find a solution to your problem in the above resources, proceed to the :ref:`Where to send? ` section. .. _getting-help-where-to-send-label: Where to send? -------------- Different types of questions and problems should be sent to different places. If you have: #. **A general end-user question or problem:** you should probably subscribe to the `Open MPI user's mailing list `_ and post it there. .. note:: Because of spam, only subscribers to the mailing list are allowed to post to the mailing list. Specifically: **you must subscribe to the mailing list before posting.** * If you have a compile-time question or problem, see the :ref:`For problems building or installing Open MPI ` section below for the content of what to include in your email. * If you have problems launching your MPI or OpenSHMEM application successfully, see the :ref:`For problems launching MPI or OpenSHMEM applications ` section below for the content of what to include in your email. * If you have other questions or problems about running your MPI or OpenSHMEM application, see the :ref:`For problems running MPI or OpenSHMEM applications ` section below for the content of what to include in your email. .. important:: The more information you include in your report, the better. E-mails/bug reports simply stating, "It doesn't work!" are not helpful; we need to know as much information about your environment as possible in order to provide meaningful assistance. **The best way to get help** is to provide a "recipe" for reproducing the problem. This will allow the Open MPI developers to see the error for themselves, and therefore be able to fix it. .. important:: Please **use a descriptive "subject" line in your email!** Some Open MPI question-answering people decide whether to read a mail based on its subject line (e.g., to see if it's a question that they can answer). So please *please* **please** use a good subject line that succinctly describes your problem. #. **A bug report:** you should probably post it to `Open MPI's Github issue tracker `_. Follow the template to submit all the requested information. #. **A patch, bug fix, or other code submission:** please post a Github Pull Request to the `Open MPI Github repository `_. #. **A developer-level / internal question about Open MPI itself:** you should probably subscribe to the `Open MPI developer's mailing list `_ and post it there. If you're unsure where to send your question, subscribe and send an email to the user's mailing list (i.e., option #1, above). .. _getting-help-compile-time-label: For problems building or installing Open MPI -------------------------------------------- If you cannot successfully configure, build, or install Open MPI, please provide *all* of the following information: #. The version of Open MPI that you're using. #. The stdout and stderr from running ``configure``. #. All ``config.log`` files from the Open MPI build tree. #. Output from when you ran ``make V=1 all`` to build Open MPI. #. Output from when you ran ``make install`` to install Open MPI. The script below may be helpful to gather much of the above information (adjust as necessary for your specific environment): .. code-block:: bash #!/usr/bin/env bash set -euxo pipefail # Make a directory for the output files dir="`pwd`/ompi-output" mkdir $dir # Fill in the options you want to pass to configure here options="" ./configure $options 2>&1 | tee $dir/config.out tar -cf - `find . -name config.log` | tar -x -C $dir # Build and install Open MPI make V=1 all 2>&1 | tee $dir/make.out make install 2>&1 | tee $dir/make-install.out # Bundle up all of these files into a tarball filename="ompi-output.tar.bz2" tar -jcf $filename `basename $dir` echo "Tarball $filename created" Then attach the resulting ``ompi-output.tar.bz2`` file to your report. .. caution:: The mailing lists have **a 150 KB size limit on messages** (this is a limitation of the mailing list web archives). If attaching the tarball makes your message larger than 150 KB, you may need to post the tarball elsewhere and include a link to that tarball in your mail to the list. .. _getting-help-launching-label: For problems launching MPI or OpenSHMEM applications ---------------------------------------------------- If you cannot successfully launch simple applications across multiple nodes (e.g., the non-MPI ``hostname`` command, or the MPI "hello world" or "ring" sample applications in the ``examples/`` directory), please provide *all* of the information from the :ref:`For problems building or installing Open MPI ` section, and *all* of the following additional information: #. The output of the ``ompi_info --all`` command from the node where you are invoking :ref:`mpirun(1) `. #. If you have questions or problems about process mapping or binding, send the output from running the ``lstopo -v`` and ``lstopo --of xml`` commands from a recent version of `Hwloc `_. #. If running on more than one node, also include the output of the ``ompi_info --version`` command **from each node on which you are trying to run**. #. The output of running ``mpirun --map-by ppr:1:node --prtemca plm_base_verbose 100 --prtemca rmaps_base_verbose 100 --display alloc hostname``. Add in a ``--hostfile`` argument if needed for your environment. The script below may be helpful to gather much of the above information (adjust as necessary for your specific environment). .. note:: It is safe to run this script after running the script from the :ref:`building and installing ` section. .. code-block:: bash #!/usr/bin/env bash set -euxo pipefail # Make a directory for the output files dir="`pwd`/ompi-output" mkdir -p $dir # Get installation and system information ompi_info --all 2>&1 | tee $dir/ompi-info-all.out lstopo -v | tee $dir/lstopo-v.txt lstopo --of xml | tee $dir/lstopo.xml # Have a text file "my_hostfile.txt" containing the hostnames on # which you are trying to launch for host in `cat my_hostfile.txt`; do ssh $host ompi_info --version 2>&1 | tee $dir/ompi_info-version-$host.out ssh $host lstopo -v | tee $dir/lstopo-v-$host.txt ssh $host lstopo --of xml | tee $dir/lstopo-$host.xml done # Have a my_hostfile.txt file if needed for your environment, or # remove the --hostfile argument altogether if not needed. set +e mpirun \ --hostfile my_hostfile.txt \ --map-by ppr:1:node \ --prtemca plm_base_verbose 100 \ --prtemca rmaps_base_verbose 100 \ --display alloc \ hostname 2>&1 | tee $dir/mpirun-hostname.out # Bundle up all of these files into a tarball filename="ompi-output.tar.bz2" tar -jcf $filename `basename $dir` echo "Tarball $filename created" .. _getting-help-running-label: For problems running MPI or OpenSHMEM applications -------------------------------------------------- If you can successfully launch parallel MPI or OpenSHMEM applications, but the jobs fail during the run, please provide *all* of the information from the :ref:`For problems building or installing Open MPI ` section, *all* of the information from the :ref:`For problems launching MPI or OpenSHMEM applications ` section, and then *all* of the following additional information: #. A *detailed* description of what is failing. *The more details that you provide, the better.* Please include at least the following information: * The exact command used to run your application. * Any relevant MCA parameters that were set (or unset) when you ran (from either the command line, environment, parameter file, etc.). * The value of the ``PATH`` and ``LD_LIBRARY_PATH`` environment variables (did you set them correctly to point to all relevant executables, the Open MPI libraries, and any required support libraries, such as libraries required for high-speed networks such as InfiniBand). #. The source code of a short sample program (preferably in C or Fortran) that exhibits the problem. #. If you are experiencing networking problems, include detailed information about your network. .. error:: TODO Update link to IB FAQ entry. #. For RoCE- or InfiniBand-based networks, include the information :ref:`in this FAQ entry `. #. For Ethernet-based networks (including RoCE-based networks), include the output of the ``ip addr`` command (or the legacy ``ifconfig`` command) on all relevant nodes. .. note:: Some Linux distributions do not put ``ip`` or ``ifconfig`` in the default ``PATH`` of normal users. Try looking for it in ``/sbin`` or ``/usr/sbin``.