kernels and data structures (found in example: gabls1)

[turbulentphil]

Dear Neks,

starting with NekRS (v23), I had a look at the examples. The following questions are particularly motivated by example gabls1 and its kernels divStress and gradY defined in gabls.oudf. These kernels seem to be derived from $NEKRS_HOME/kernels/nrs/gradientVolumeHex3D.okl.

1. Is it possible to call such pre-existing kernels in the .udf file (other than copying them to the .oudf file)? If so, how can one achieve this?
2. The kernels divStress and gradY have one more input argument (the inverse lumped mass matrix invLMM) than gradientVolumeHex3D. I found that gradient fields obtained directly from gradientVolumeHex3D are "incorrect" unless their solution is additionally weighted by invLMM. (I write "incorrect" because in order to compute global averages of solution fields, we need to weight them typically by LMM which I also expect to be necessary for the output of the gradient function.) This raises a question about the meaning/content of the different kernel arguments (say, of gradientVolumeHex3D).

I can decipher the following:

• Nelements = number of spectral elements
• offset = fieldOffset used to separate different components of P
• P = scalar field
• gradP = resulting gradient field
• D = coefficients that result from derivatives of the local spectral basis functions (e.g., Dr)

I can not completely decipher the following:

• vgeo. What does vgeo stand for? It seems to contain the derivatives of the local coordinates {r, s, t} to the global coordinates {x, y, z} (e.g., drdx), as well as Jacobian weights (JW).

As JW seems to contain weights (and P is not yet weighted by LMM), why does invLMM become additionally necessary?

3. Returning to gabls.udf where for instance the kernel gradY is called, I find in the subsequent line a oogs::startFinish call. Why is this (gather scatter operation) necessary and what does this actually do? Isn't the solution that is returned from the kernel (o_ddyAvg, which is equivalent to gradP in the kernel function itself) already up-to-date in the device memory?
   When I compute a volume average of o_ddyAvg/gradP (subsequently weighted by LMM), I do not get the expected results without this oogs::startFinish call.

4. Maybe one last question related to fieldOffset. This quantity can be found at several places, for instance at nrs->fieldOffset, nrs->meshV->fieldOffset, nrs->cds->fieldOffset[0], or nrs->cds->mesh[0]->fieldOffset. Are fieldOffsets from the velocity or (potentially different) scalar fields guaranteed to equal or could they differ?

I acknowledge that these are a bunch of questions and a single answer might not address all of them, but I would really appreciate any help anyway.

Best,
Philipp

[stgeke]

1. The simplest approach is to use #include "<okl source>/foo.okl in the udf OKL section. Note, basic SEM ops can be found here: https://github.com/Nek5000/nekRS/blob/next/src/core/opSEM.hpp
2. vgeo stands for volumetric geometric factors
3. oogs::startFinish implements a gather (local to global) operation followed by a scatter (global to local). This is required to maintain C0 continuity with surrounding elements, the local DOFs must be assembled with the corresponding local DOFs in adjacent elements to create a continuous, global DOF.
4. The offsets can vary, but for now, they are the same.

[turbulentphil]

Thank you for your answer!

1. It’s great that these basic functions will make it into the next version of NekRS.
2. OK, so LMM is in some sense part of vgeo and thus we need to multiply by invLMM separately (given that a subsequent averaging procedure multiplies, as usual, again by LMM).
3. The way I understand your answer is the following:
   3.1 The gradient field computed by, e.g., gradY is correct in the sense that this represents the gradient field obtained from each spectral element itself.
   3.2 This gradient field is (of course) not continuous any more at the spectral element boundaries.
   3.3 The solution of the function call is already stored / up-to-date in the solution array / device memory.
   3.4 oogs::startFinish averages the values at the spectral element boundaries based on the values from the adjacent elements, thus generating a smooth continuous solution.
   3.5 So, in some sense, we have translated the solution field from being true within-element only to continuous across the individual elements.
4. OK, good to know or to be aware of!

[stgeke]

Note, invLMM != 1/LMM. Instead it refers to the inverse of the assembled mass matrix.

…

On Mon, 15 Jul 2024 at 17:19, turbulentphil ***@***.***> wrote: Thank you for your answer! 1. It’s great that these basic functions will make it into the next version of NekRS. 2. OK, so LMM is in some sense part of vgeo and thus we need to multiply by invLMM separately (given that a subsequent averaging procedure multiplies, as usual, again by LMM). 3. The way I understand your answer is the following: 3.1 The gradient field computed by, e.g., gradY is correct in the sense that this represents the gradient field obtained from each spectral element itself. 3.2 This gradient field is (of course) not continuous any more at the spectral element boundaries. 3.3 The solution of the function call is already stored / up-to-date in the solution array / device memory. 3.4 oogs::startFinish averages the values at the spectral element boundaries based on the values from the adjacent elements, thus generating a smooth continuous solution. 3.5 So, in some sense, we have translated the solution field from being true within-element only to continuous across the individual elements. 4. OK, good to know or to be aware of! — Reply to this email directly, view it on GitHub <#577 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEN2M76DQR4UXZVCFO2Z74DZMPK6FAVCNFSM6AAAAABKRGELOOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMBVGE3TSMA> . You are receiving this because you commented.Message ID: ***@***.***>