Programming parallel for C++ and Python application

ETC High Performance Computing
1

I am looking to have a C++ and Python application developed using the processing power of at least one GPU.
We are already using Boost, so Boost.MPI comes to mind.
We already have some experience in developing C++ applications for multi-core CPUs by using threading.
However, since we have not developed sufficient expertise in MPI, we were wondering if a Python solution would be more appropriate, because it will give the developers a better understanding of what the code does. In such a case something like mpi4py would come to mind.

Of course having a GPU specifically for simulation purposes could mean that something like PyOpenCL may be even better.
I mention "simulation", because the machine will be calculating variations on what is largely the same problem, ie not be a general purpose machine.

So in summary:
1- For raw processing speed with a multi-core CPU and a GPU, would a openCL solution or Boost.MPI solution be recommended?
2- Should the Python solution be used only for high level functionality, such as handling data upon return, or does Python compromise speed too much?

2

What you have basically in both Boost and mpi4py is just embedded Openmpi.
IMO since mpi is straightforward (mostly just pragmas) in C/C++ you can tweak things
merely by either recompiling with -D to define some things or actually read in command line arguments.

See here for libraries and documentation:
Open MPI: Open Source High Performance Computing

3

Thank you for your response. What this likely will come down to is setting up the core multi-core calculations in Boost and do some post-processing in Python. The post-processing will take place at a higher level (ie user) and is likely to be more prone to changes.

4

I am getting concerned that our compiled Boost-code is getting bloated. We will be experimenting with basic C++ functions as if there was only one core and no Boost, then add MPI pragmas to the C++ code and finally add user functionality (screens, post-processing etc) in Python.