Basic Usage

Sample configuration files and geometry input files are provided with the binary to test the proper system setting and see if the simulation starts correctly on the system.

Single and Double Precision Binaries

Since v1.4, nanoFluidX has being delivered in both single (SP) and double (DP) precision. It has been noted that for some very specific applications/simulations the single precision is insufficient and double precision should be used. It is difficult to universally predict in which cases there is a need for double precision, therefore single precision binary contains a warning message with some useful input about possible precision issues. It is up to you to acknowledge the warning message and decide whether it is necessary to use double precision or not.

As an example of such a situation, one can imagine a case where there are large discrepancies in scales for both velocity and positions (for example, max velocity of 30 m/s and the need to accurately resolve movement with min velocity of 0.0003 m/s).

We do however find single precision to be of satisfactory accuracy for most realistic cases.

Performance of single Vs. double precision binaries varies greatly depending on the type of GPU, but overall, it is realistic to expect at least two times slower run times when using double precision. Latest generation of NVIDIA Tesla GPU’s (the V100) significantly improved both single and double precision performance.

NVIDIA SMI

Similar to the command “top” to see the currently running jobs on the CPU, NVIDIA provides the tool “nvidia-smi” that shows the actual usage of the GPUs. Also, the currently occupied GPU’s can be detected if already some jobs are running on that machine.

Launching a Simulation

The command to start a simulation is:

$ CUDA_VISIBLE_DEVICES=0 mpiexec –np 1 nanoFluidX_HW_2020.1_SP.bin –i test.cfg

The first command (CUDA_VISIBLE_DEVICES) chooses the GPU-devices that are used to start this job. By usage of this keyword it is possible to avoid running different jobs on the same card. Invoking a simulation without this command will always try to run the job on device number zero! For multiple cards just list the device numbers separated with a comma and set the number of “processes” to “-np #N”. Ensure the number of processes -np matches the number of selected GPU's. For example, if you want to run on 3 GPU's, the command should look like:

$ CUDA_VISIBLE_DEVICES=0,1,2 mpiexec –np 3 nanoFluidX_HW_2020.1_SP.bin –i test.cfg

Note: The CPU is used for nanoFluidX to control the simulation, the actual work is done on the GPU.

To launch a simulation, a configuration file has to be passed to the binary via the option “-i”, followed by the configuration file name. This file and the corresponding geometry file need to be placed in the same folder where the simulation is started.

Now, please try to simulate one of the example cases provided with the binary.