Fluent-UDF-14.5-L07-Parallel

©2012ANSYS,Inc.June17,20131Release14.514.5ReleaseChapter7UDFsforCalculationsinParallelUserDefinedFunctionsinANSYSFluent©2012ANSYS,Inc.June17,20132Release14.5ParallelFluentArchitecture•Hostprocessislabeled999999•Cortexisthefrontendprocesswhichconnectsonlytothehostprocess•ComputeNodeprocessesarelabeledconsecutivelystartingat0•EachcomputenodeisvirtuallyconnectedtoeveryothercomputenodebytheMessagePassing(MP)interface©2012ANSYS,Inc.June17,20133Release14.5CommandTransferinParallelFluent“(%iterate5)”CortexHostCompute-Node-0Compute-Node-1Compute-Node-2Compute-Node-3“(%iterate5)”•HostcommunicateswithNode0•Node0communicateswithallothernodes•AllNodes(=1)communicateswithhostthroughNode0scheme©2012ANSYS,Inc.June17,20134Release14.5ParallelizingaSerialUDFcode•Ingeneral,UDFcodesneedtobeparallelizedinordertomakesurethecodesareexecutedproperlyaseitherahostornodeprocess•ToparallelizeaUDFweneedtomodifyaserialcodesuchthatitwillrunsuccessfullyonbothserialandparallelmodes•Weusecompilerdirectives(preprocessorcommands)insertedintotheoriginalCcodetodistinguishwhichpartoftheprogramisexecutedbythehostorbythenodeswhentheUDFiscompiled•Thesyntaxofthe#ifcompilerdirectiveisquitesimple:#ifisacompilerdirective(similarto#define)#endifisusedtoclosean#if©2012ANSYS,Inc.June17,20135Release14.5BasicCompilerDirectives#ifRP_HOST/*CodinghereonlyperformedonHOSTprocess*/#endif#ifRP_NODE/*CodinghereonlyperformedonNODEprocesses*/#endif#ifPARALLEL/*CodinghereonlyperformedonHOST&NODEprocesses*/#endifDifferentpartsoftheUDFcodehavetobeperformedoneitherthehostornodeprocesses.Thesepartscanbeselectedusing:©2012ANSYS,Inc.June17,20136Release14.5NegatedCompilerDirectives#if!PARALLEL/*CodinghereonlyperformedonSERIALprocess*/#endif#if!RP_HOST/*CodinghereonlyperformedonNODE&SERIALprocesses*/#endif#if!RP_NODE/*CodinghereonlyperformedonHOST&SERIALprocesses*/#endifSincemanyoftheoperationswillalsoberequiredintheserialversion,thenegatedversionsaremorecommonlyused:©2012ANSYS,Inc.June17,20137Release14.5Partitioning(1)•DomainDecomposition:SplitsthecellsinthedomainacrossComputeNodes•BecauseFluent’salgorithmsexpectacelltobeonbothsidesofaninteriorface,copiesoftheneighboringpartition’scellsarekeptoneachNode•ComputeNode0hascopiesofthecellsontheothersideofallpartitionfacesandComputeNode1hascorrespondingcellcopiesfromNode0DomainDecompositionComputeNode0ComputeNode1DistributionacrossComputeNodes©2012ANSYS,Inc.June17,20138Release14.5Partitioning(2)•Themaincellsofeachpartitionaredesignatedas“Interior”cellsandtheadditionalcopiedcellsfromotherComputeNodesaredesignatedas“Exterior”cells.ThePartitionBoundaryFacesareaspecialtypeofInteriorfacePartitionBoundaryFaceInteriorFacesExteriorCellComputeNode0SurfaceBoundaryZoneFaceInteriorCells©2012ANSYS,Inc.June17,20139Release14.5PartitionedThreadLoop(1)•Theloopmacroforcellsinserialis:begin_c_loop(c,t){}end_c_loop(c,t)•Inparallel,thisserialformloopsthroughbothinteriorandexteriorcells•Usebegin_c_loop_int(c,t)inUDFsthataretobeparallelized:begin_c_loop_int(c,t){}end_c_loop_int(c,t)•Thisloopexcludestheexteriorcellssotheydon’tgetvisitedtwice.•Anotherloopconstructloopsthroughtheexteriorcellsonly:begin_c_loop_ext(c,t){}end_c_loop_ext(c,t)butitisrarelyusedinUDFsanddoesnothingifcompiledintheserialversion©2012ANSYS,Inc.June17,201310Release14.5PartitionedThreadLoop(2)•Similarloopsexistforfaces:begin_f_loop_all(f,t){…}end_f_loop_all(f,t)begin_f_loop_int(f,t){…}end_f_loop_int(f,t)•Butthesehaveaslightlydifferentusesoinstead,usethestandardloopandchecktoseeifthefaceisallocatedtothisnodebyusingPRINCIPAL_FACE_P(f,t):begin_f_loop(f,t){if(PRINCIPAL_FACE_P(f,t)){…Onlygetsdoneonceperface…}}end_f_loop(f,t)©2012ANSYS,Inc.June17,201311Release14.5Inter-ProcessCommunication(1)•Eachcomputenodemaintainlocalcacheofindividualvariables•Synchronizationormakeglobalreductionofsuchdatainvolvescommunicationinaparticularorder•ConsiderthesimpleoperationofpassingauserdefinedcortexparameterthatissetusingschemebutisusedinaUDFSerialVersion#include“udf.h“DEFINE_INIT(init_temp,domain){realinit_temp;/*Variabledeclaration*/Thread*ct;cell_tc;init_temp=RP_Get_Real(“my-init-temp“);/*Schemecommunication*/thread_loop_c(ct,domain)/*Solutiondatahandling*/begin_c_loop(c,ct){C_T(c,ct)=init_temp;}end_c_loop(c,ct)}©2012ANSYS,Inc.June17,201312Release14.5Inter-ProcessCommunication(2)Combined(Serial&Parallel)Code#include“udf.h“DEFINE_INIT(init_temp,domain){realinit_temp;/*Variabledeclaration.Somevariablesusedinallversions.*/#if!RP_HOSTThread*ct;/*Thesevariableonlyusedonthenodesandinserial*/cell_tc;#endif#if!RP_NODEinit_temp=RP_Get_Real(“my-init-temp“);/*Schemecommunicationonhostandserial*/#endifhost_to_node_real_1(init_temp);/*Hostsendsdatatonodes(Doesnothinginserial)*/#if!RP_HOSTthread_loop_c(ct,domain)/*Solutiondatahandlingonlyonnodesandserial*/begin_c_loop(c,ct){C_T(c,ct)=init_temp;}end_c_loop(c,ct)#endif}©2012ANSYS,Inc.June17,201313Release14.5Inter-ProcessCommunication(3)•Themacrohost_to_node_real_1(init_temp)isdefinedasaSendcommandintheHostversionandaReceivecommandintheComputeNodes.ItdoesnothingintheSer