For detailed documentation of the OpenFPM sources, including the examples, see the online Doxygen documentation.

## Vector 0: Simple vector initialization

This example show several basic functionalities of the distributed vector `vector_dist`

.
The distributed vector is a set of particles in an N-dimensional space.
In this example it is shown how to:

- Initialize the library
- Create a Box that defines the domain
- An array that defines the boundary conditions
- A Ghost object that will define the extension of the ghost part in physical units

*The source code of the example Vector/0_simple/main.cpp. The full doxygen documentation Vector_0_simple*.

See also our video lectures dedicated to this topic Video 1, Video 2

## Example 1: Vector Ghost layer

This example shows the properties of `ghost_get`

and `ghost_put`

- functions
that synchronize the ghosts layer for a distributed vector `vector_dist`

.

In this example it is shown how to:

- Iterate
`vector_dist`

via`getDomainIterator`

- Redistribute the particles in
`vector_dist`

according to the underlying domain decomposition via`map`

- Synchronize the ghost layers in the standard way
`NO_POSITION`

,`KEEP_PROPERTIES`

and`SKIP_LABELLING`

options of the`ghost_get`

function- Propagate the data from ghost to non-ghost particles via
`ghost_put`

*The source code of the example Vector/1_ghost_get_put/main.cpp. The full doxygen documentation Vector_1_ghost_get*.

## Example 2: Cell-lists and Verlet-lists

This example shows the properties of `ghost_get`

and `ghost_put`

- functions
that synchronize the ghosts layer for a distributed vector `vector_dist`

.

Key points:

- How to utilize the grid iterator
`getGridIterator`

, to create a grid-like particle domain - Two principal types of fast neighbor lists: cell-list
`getCellList`

and Verlet-list`getVerlet`

for a distributed vector`vector_dist`

`CELL_MEMFAST`

,`CELL_MEMBAL`

and`CELL_MEMMW`

variations of the cell-list, with different memory requirements and computations costs- Iterating through the neighboring particles via
`getNNIterator`

of cell-list and Verlet-list

*The source code of the example Vector/1_celllist/main.cpp. The full doxygen documentation Vector_1_celllist*.

## Example 3: GPU vector

This example shows how to create a vector data-structure with `vector_dist_gpu`

to access a `vector_dist`

-alike data structure from GPU accelerated computing code.

Key points:

- How to convert the source code from using
`vector_dist`

to`vector_dist_gpu`

and how it influences the memory layout of the data structure - Oflloading particle position
`hostToDevicePos`

and particle property`hostToDeviceProp`

data from CPU to GPU - Lanuching a CUDA-like kernel with
`CUDA_LAUNCH`

and automatic subdivision of a computation loop into workgroups/threads via`getDomainIteratorGPU`

or manually specifying the number of workgroups and the number of threads in a workgroup - Passing the data-structures to a CUDA-like kernel code via
`toKernel`

- How to use
`map`

with the option`RUN_DEVICE`

to redistribute the particles directly on GPU, and`ghost_get`

with`RUN_DEVICE`

option to fill ghost particles directly on GPU - How to detect and utilize RDMA on GPU to get the support of CUDA-aware MPI implementation to work directly with device pointers in communication subroutines

*The source code of the example Vector/1_gpu_first_step/main.cpp. The full doxygen documentation Vector_1_gpu_first_step*.

## Example 4: HDF5 Save and load

This example show how to save and load a vector to/from the parallel file format HDF5.

Key points:

- How to save the position/property information of the particles
`vector_dist`

into an*.hdf5*file via`save`

- How to load the position/property information of the particles
`vector_dist`

from an*.hdf5*file via`load`

*The source code of the example Vector/1_HDF5_save_load/main.cpp. The full doxygen documentation Vector_1_HDF5*.

## Example 5: Vector expressions

This example shows how to use vector expressions to apply mathematical operations and functions on particles.
The example also shows to create a point-wise applicable function

$$ A_q e^{\frac{|x_p-x_q|^2}{\sigma}} $$

where $A_q$ is the property $A$ of particle $q$, $x_p, x_q$ are positions of particles $p, q$ correspondingly.

Key points:

- Setting an alias for particle properties via
`getV`

of`particle_dist`

to be used within an expression - Composing expressions with scalar particle properties
- Composing expressions with vector particle properties. The expressions are 1) applied point-wise; 2) used to create a component-wise multiplication via
`*`

; 3) scalar product via`pmul`

; 4) compute a norm`norm`

; 5) perform square root operation`sqrt`

- Converting
`Point`

object into an expression`getVExpr`

to be used with vector expressions - Utilizing
`operator=`

and the function`assign`

to assing singular or multiple particle properties per iteration through particles - Constructing expressions with
`applyKernel_in`

and`applyKernel_in_gen`

to create kernel functions called at particle locations for all the neighboring particles, e.g. as in SPH

$$\sum_{q = Neighborhood(p)} A_q D^{\beta}ker(x_p,x_q) V_q $$

*The source code of the example Vector/2_expressions/main.cpp. The full doxygen documentation Vector_2_expression*.

## Example 6: Molecular Dynamics with Lennard-Jones potential (Cell-List)

This example shows a simple Lennard-Jones molecular dynamics simulation in a stable regime.
The particles interact with the interaction potential

$$ V(x_p,x_q) = 4( (\frac{\sigma}{r})^{12} - (\frac{\sigma}{r})^6 ) $$

$A_q$ is the property $A$ of particle $q$, $x_p, x_q$ are positions of particles $p, q$ correspondingly, $\sigma$ is a free parameter, $r$ is the distance between the particles.

Key points:

- Reusing memory allocated with
`getCellList`

for the subsequent iterations via`updateCellList`

- Utilizing
`CELL_MEMBAL`

with`getCellList`

to minimize memory footprint - Performing 10000 time steps using symplectic Verlet integrator

$$ \vec{v}(t_{n}+1/2) = \vec{v}_p(t_n) + \frac{1}{2} \delta t \vec{a}(t_n) $$

$$ \vec{x}(t_{n}+1) = \vec{x}_p(t_n) + \delta t \vec{v}(t_n+1/2) $$

$$ \vec{v}(t_{n+1}) = \vec{v}_p(t_n+1/2) + \frac{1}{2} \delta t \vec{a}(t_n+1) $$

- Producing a time-total energy 2D plot with
`GoogleChart`

*The source code of the example Vector/3_molecular_dynamic/main.cpp. The full doxygen documentation Vector_3_md_dyn*.

## Example 7: Molecular Dynamics with Lennard-Jones potential (Verlet-List)

The physical model in the example is identical to Molecular Dynamics with Lennard-Jones potential (Cell-List). Please refer to it for futher details. Key points:

- Due to the computational cost of updating Verlet-list,
*r_cut + skin*cutoff distance is used such that the Verlet-list has to be updated once in 10 iterations via`updateVerlet`

- As Verlet-lists are constructed based on local particle id's, which would be invalidated by
`map`

or`ghost_get`

,`map`

is called every 10 time-step, and`ghost_get`

is used with`SKIP_LABELLING`

option to keep old indices every iteration

*The source code of the example Vector/3_molecular_dynamic/main_vl.cpp. The full doxygen documentation Vector_3_md_vl*.

## Example 15: Molecular Dynamics with Lennard-Jones potential (Symmetric Verlet-List)

This example is an extension to Molecular Dynamics with Lennard-Jones potential (Verlet-List). It shows how better performance can be achieved for symmetric interaction models with symmetric Verlet-list compared to the standard Verlet-list. Key points:

- Computing the interaction for particles
*p*,*q*only once - Propagate the data from potentially ghost particles
*q*to non-ghost particles in their corresponding domains via`ghost_put`

with the operation`add_`

- Changing the prefactor in the subroutine of calculating the total energy as every pair of particles is visited once (as compared to two times before)
- Updating Verlet-list once in 10 iterations via
`updateVerlet`

with 'VL_SYMMETRIC' flag

*The source code of the example Vector/5_molecular_dynamic_sym/main.cpp. The full doxygen documentation Vector_5_md_vl_sym*.

## Example 16: Molecular Dynamics with Lennard-Jones potential (Symmetric CRS Verlet-List)

This example is an extension to Molecular Dynamics with Lennard-Jones potential (Verlet-List) and Molecular Dynamics with Lennard-Jones potential (Verlet-List). It shows how better performance can be achieved for symmetric interaction models with symmetric Verlet-list compared to the standard Verlet-list. Key points:

- Computing the interaction for particles
*p*,*q*only once - Propagate the data from potentially ghost particles
*q*to non-ghost particles in their corresponding domains via`ghost_put`

with the operation`add_`

- Changing the prefactor in the subroutine of calculating the total energy as every pair of particles is visited once (as compared to two times before)
- Updating Verlet-list once in 10 iterations via
`updateVerlet`

with 'VL_SYMMETRIC' flag

*The source code of the example Vector/5_molecular_dynamic_sym/main.cpp. The full doxygen documentation Vector_5_md_vl_sym*.

## Example 8: Molecular Dynamics with Lennard-Jones potential (GPU)

The physical model in the example is identical to Molecular Dynamics with Lennard-Jones potential (Cell-List) and Molecular Dynamics with Lennard-Jones potential (Verlet-List). Please refer to those for futher details. Key points:

- To get the particle index inside a CUDA-like kernel
`GET_PARTICLE`

macro is used to avoid overflow in the construction`blockIdx.x * blockDim.x + threadIdx.x`

- A primitive reduction function
`reduce_local`

with the operation`_add_`

is used to get the total energy by summing energies of all particles.

*The source code of the example Vector/3_molecular_dynamic_gpu/main_vl.cpp. The full doxygen documentation Vector_3_md_dyn_gpu*.

## Example 9: Molecular Dynamics with Lennard-Jones potential (GPU optimized)

The physical model in the example is identical to Molecular Dynamics with Lennard-Jones potential (Cell-List), Molecular Dynamics with Lennard-Jones potential (Verlet-List) and is based on Molecular Dynamics with Lennard-Jones potential (GPU). Please refer to those for futher details. Key points:

- To achieve coalesced memory access on GPU and to reduce cache load the particle indices are stored in cell-list in a sorted manner, i.e. particles with neighboring indices are located in the same cell-list. This is achieved by assigning new particle indices and storing them temporarily in
`vector_dist`

. - The sorted version of
`vector_dist_gpu`

is offloaded to GPU using`toKernel_sorted`

. It uses`get_sort`

instead of`get`

to get a particle index in the cell-list neighborhood iterator`getNNIteratorBox`

- The sorted version of particle properties have to be merged to the original ones once the processing is done via
`merge_sort`

of`vector_dist`

*The source code of the example Vector/3_molecular_dynamic_gpu_opt/main_vl.cpp. The full doxygen documentation Vector_3_md_dyn_gpu_opt*.

## Example 10: Molecular Dynamics with Lennard-Jones potential (Particle reordering)

The physical model in the example is identical to Molecular Dynamics with Lennard-Jones potential (Cell-List), Molecular Dynamics with Lennard-Jones potential (Verlet-List). The example shows how reordering the data can significantly reduce the computational running time. Key points:

- The particles inside
`vector_dist`

are reordered via`reorder`

following a Hilbert curve of order*m*(here*m=5*) passing through the cells of $2^m \times 2^m \times 2^m$ (here, in 3D) cell-list - It is shown that the frequency of reordering depends on the mobility of particles
- Wall clock time is measured of the function
`calc_force`

utilizing the object`timer`

via`start`

and`stop`

*The source code of the example Vector/4_reorder/main_data_ord.cpp. The full doxygen documentation Vector_4_reo*.

## Example 11: Molecular Dynamics with Lennard-Jones potential (Cell-list reordering)

The physical model in the example is identical to Molecular Dynamics with Lennard-Jones potential (Cell-List), Molecular Dynamics with Lennard-Jones potential (Verlet-List). The example shows how reordering the data can significantly reduce the computational running time. Key points:

- The cell-list cells are iterated following a Hilbert curve instead of a normal left-to-right bottom-to-top cell iteration (in 2D). The function
`getCellList_hilb`

of`vector_dist`

is used instead of`getCellList`

- It is shown that for static or slowly moving particles a speedup of up to 10% could be achieved

*The source code of the example Vector/4_reorder/main_comp_ord.cpp. The full doxygen documentation Vector_4_comp_reo*.

## Example 12: Complex properties in Vector 1

This example shows how to use complex properties in the distributed vector `vector_dist`

Key points:

- Creating a distributed vector with particle properties: scalar, vector
`float[3]`

,`Point`

, list of float`openfpm::vector<float>`

, list of custom structures`openfpm::vector<A>`

(where`A`

is a user-defined type with no pointers), vector of vectors`openfpm::vector<openfpm::vector<float>>>`

- Redistribute the particles in
`vector_dist`

according to the underlying domain decomposition. Communicate only the selected particle properties via`map_list`

(instead of communicating all`map`

) - Synchronize the ghost layers only for the selected particle properties
`ghost_get`

*The source code of the example Vector/4_complex_prop/main.cpp. The full doxygen documentation Vector_4_complex_prop*.

## Example 13: Complex properties in Vector 2

This example shows how to use complex properties in the distributed vector `vector_dist`

Key points:

- Creating a distributed vector with particle properties: scalar, vector
`float[3]`

,`Point`

, list of float`openfpm::vector<float>`

, list of custom structures`openfpm::vector<A>`

(where`A`

is a user-defined type with memory pointers inside), vector of vectors`openfpm::vector<openfpm::vector<float>>>`

- Enabling the user-defined type being serializable by
`vector_dist`

via`packRequest`

method to indicate how many byte are needed to serialize the structure`pack`

method to serialize the data-structure via methods`allocate`

,`getPointer`

of`ExtPreAlloc`

and method`pack`

of`Packer`

`unpack`

method to deserialize the data-structure via method`getPointerOffset`

of`ExtPreAlloc`

and method`unpack`

of`Unpacker`

`noPointers`

method to inform the serialization system that the object has pointers- Constructing constructor, destructor and
`operator=`

to avoid memory leaks

*The source code of the example Vector/4_complex_prop/main.cpp. The full doxygen documentation Vector_4_complex_prop_ser*.

## Example 14: Multiphase Cell-lists and Verlet-lists

This example is an extension to Example 2: Cell-lists and Verlet-lists and ()[]. It shows how to use multi-phase cell-lists and Verlet-list using multiple instances of `vector_dist`

.
Key points:

- All the phases have to use the same domain decomposition, which is achieved by passing the decomposition of the first phase to the constructor of
`vector_dist`

of all the other phases. - The domains have to be iterated individually via
`getDomainIterator`

, the particles redistributed via`map`

, the ghost layers synchronized via`ghost_get`

for all the phases`vector_dist`

. - Constructing Verlet-lists for two phases (
*ph0*,*ph1*) with`createVerlet`

, where for one phase*ph0*the neighoring particles of*ph1*are assigned in the Verlet-list. Cell-list of*ph1*has to be passed to`createVerlet`

- Constructing Verlet-lists for multiple phases (
*ph0*,*ph1*,*ph2*...) with`createVerletM`

, where for one phase*ph0*the neighoring particles of*ph1*,*ph2*... are assigned in the Verlet-list. Cell-list containing all of*ph1*,*ph2*... create with`createCellListM`

has to be passed to`createVerletM`

- Iterating over the neighboring particles of a multiphase Verlet-list with
`getNNIterator`

with`get`

being substituded by`getP`

(particle phase) and`getV`

(particle id) - Extending example of the symmetric interaction for multiphase cell-lists and Verlet-lists via
`createCellListSymM`

,`createVerletSymM`

*The source code of the example Vector/4_multiphase_celllist_verlet/main.cpp. The full doxygen documentation Vector_4_mp_cl*.

## Example 16: Validation and debugging

This example shows how the flexibility of the library can be used to perform complex tasks for validation and debugging. Key points:

- To get unique global id's of the particles the function
`accum`

of`vector_dist`

is used, which returns prefix sum of local domain sizes $j<i$ for the logical processor $i$ out of $N$ total processors - Propagate the data from potentially ghost particles
*q*to non-ghost particles in their corresponding domains via`ghost_put`

with the operation`merge_`

, that merges two`openfpm::vector`

(ghost and non-ghost)

*The source code of the example Vector/6_complex_usage/main.cpp. The full doxygen documentation Vector_6_complex_usage*.