Many SAP Commerce projects are still using regular storefront approach, often incorporating modern frameworks like Vue.js and using default SAP Commerce build. Implementing parallel execution of FE and BE builds can lead to significant reductions in overall build time. Even for projects utilizing the OOTB Grunt build, parallelization can result in significant time savings. Furthermore, parallelization provides the valuable advantage of decoupling FE and BE builds, enabling developers to skip FE builds when focusing on BE development.
Technical Solution
The main idea of the solution is to utilize the forget ant task from the ant-contrib and use properties to track the start and completion of parallel builds. We can leverage regular properties as they are immutable and global (subsequent value change would be ignored, and once property is set, it would be available everywhere). To check completion of parallel FE build would be used waitfor ant task with isset property check.
Another implementation idea is to run parallel build only if it is FE only build or ant clean all execution, what also means that there is a sense to add additional ant targets, which would allow to run only BE or only FE build.
Implementation Details
Ant macro to run FE build asynchronously
Firstly we need macrodef, which would decide if FE build should be started and start it asynchronously. The macro would have 2 parameters:
feBaseDir - directory with package.json, where npm commands are executed. Usually, it is inside web subfolder of storefront extension.feOutDir - directory, where FE build artifacts would be generated. Usually it is _ui folder placed in webroot/WEB-INF/_ui of storefront extension. For FE source code usually is used _ui-src folder placed in webroot/WEB-INF/_ui-src of storefront extension.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
<macrodef name="build_fe_common_async">
<attribute name="feBaseDir"/>
<attribute name="feOutDir" default="NOT_SET"/>
<sequential>
<if>
<!-- ! Skip FE build if no targets fe/build_fe are present or if is not "ant clean all" directly -->
<!-- We need that dirty hack with skipping, because on CCv2 we execute "ant clean all" and we can't modify dependencies for "all" target. -->
<or>
<contains string="${ant.project.invoked-targets}" substring="fe"/>
<contains string="${ant.project.invoked-targets}" substring="build_fe"/>
<equals arg1="${ant.project.invoked-targets}" arg2="clean,all"/>
</or>
<then>
<property name="fe.build.started" value="true"/>
<echo message="*************************** TRIGGER ASYNC FE REBUILD ***************************"/>
<forget>
<build_fe_common feBaseDir="@{feBaseDir}" feOutDir="@{feOutDir}"/>
</forget>
</then>
<else>
<echo message="*************************** SKIP ASYNC FE REBUILD ***************************"/>
</else>
</if>
</sequential>
</macrodef>
|
ant.project.invoked-targets property is used to check if were executed exactly ant clean all. Such tricky check is need, as there is no possibility to change execution targets on CCv2 (ant clean all is executed there) and there is no possibility to extend all command without OOTB code modifications. This check also incorporates check on ant fe and build_fe targets, which is useful for implementing own ant targets to trigger only FE or only BE build.
After that check fe.build.started property is set, it would be needed later to not wait for async FE build completion, if it was never started. And the last step is to trigger and forget macrodef build_fe_common, which should execute FE build itself.
Common ant macro to trigger FE build execution
The build_fe_common ant macrodef is used to execute npm commands in proper folder. Similar to previous macrodef it also has 2 parameters feBaseDir and feOutDir.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
<macrodef name="build_fe_common">
<attribute name="feBaseDir"/>
<attribute name="feOutDir" default="NOT_SET"/>
<sequential>
<repairNpmMacro/>
<property name="npmInstallArguments" value="install --unsafe-perm"/>
<property name="npmBuildArguments" value="run build"/>
<echo message="FE base dir: @{feBaseDir}"/>
<echo message="************************** SHOW NODE VERSION ***************************"/>
<invokeNodeInCustomDir arguments="-v" workingDir="@{feBaseDir}"/>
<echo message="*************************** START FE REBUILD ***************************"/>
<if>
<not>
<equals arg1="@{feOutDir}" arg2="NOT_SET"/>
</not>
<then>
<echo message="delete @{feOutDir}"/>
<delete dir="@{feOutDir}" failonerror="false"/>
</then>
<else>
<echo message="Delete directory wasn't specified"/>
</else>
</if>
<trycatch>
<try>
<invokeNpmInCustomDir arguments="${npmInstallArguments}" workingDir="@{feBaseDir}"
resultproperty="fe.install.exit.code"/>
<invokeNpmInCustomDir arguments="${npmBuildArguments}" workingDir="@{feBaseDir}"
resultproperty="fe.build.exit.code"/>
</try>
<finally>
<property name="fe.build.ended" value="true"/>
</finally>
</trycatch>
<echo message="*********************** END FE REBUILD *******************************"/>
</sequential>
</macrodef>
|
First of all repairNpmMacro should be executed to ensure that bundled Node.js and npm are available. After that result directory is cleaned up, and lastly npm install and npm run build should be triggered. (npm ci can be used instead of npm run).
fe.build.ended property is set after all npm commands are finished. That property is needed to check if FE build is finished before finishing ant build itself.
Implementation uses bundled in SAP Commerce versions of Node.js and npm. For that are utilized macrodefs from npmancillary extension:
repairnpm setups proper permissions on Node.js and npm binaries inside SAP CommerceinvokeNpm runs npm commandinvokeNode runs node command
Unfortunately, it is impossible to reuse this commands directly, so it is required to copy-paste them with different namings. In example implementation would be used improved versions of repairNpmMacro, invokeNodeInCustomDir and invokeNpmInCustomDir, implementation of which could be found in Appendix.
This is just an example of most common implementation with npm build. It can be easily changed with usage of grunt/gulp/etc. instead of npm.
Ant macrodef to wait for FE build completion
The wait_fe_build_async ant macrodef is utilized to wait for the asynchronous completion of the frontend (FE) build process.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
|
<macrodef name="wait_fe_build_async">
<sequential>
<if>
<isset property="fe.build.started"/>
<then>
<!-- Wait for FE build to finish. -->
<if>
<isset property="fe.build.ended"/>
<then>
<echo message="*********************** SKIP WAITING FOR ASYNC FE BUILD *******************************"/>
</then>
<else>
<echo message="*********************** WAITING FOR ASYNC FE BUILD *******************************"/>
<if>
<property name="fe.build.async.wait.min" value="15"/>
</if>
<echo message="Will wait up to ${fe.build.async.wait.min} minutes for FE build."/>
<waitfor checkevery="5" checkeveryunit="second" maxwait="${fe.build.async.wait.min}"
maxwaitunit="minute">
<or>
<isset property="fe.build.ended"/>
</or>
</waitfor>
<if>
<isset property="fe.build.ended"/>
<then>
<echo message="*********************** END WAITING FOR ASYNC FE BUILD *******************************"/>
</then>
<else>
<fail message="FE build didn't finished in 15 minutes."/>
</else>
</if>
</else>
</if>
<!-- Check FE build state and fail whole build in case anything in FE failed. -->
<if>
<not>
<equals arg1="${fe.build.exit.code}" arg2="0"/>
</not>
<then>
<echo message="FE build FAILED with exit code ${fe.build.exit.code}. Please, check NPM logs."/>
<fail message="FE build FAILED."/>
</then>
</if>
<if>
<not>
<equals arg1="${fe.install.exit.code}" arg2="0"/>
</not>
<then>
<echo message="FE build FAILED with exit code ${fe.install.exit.code}. Please, check NPM logs."/>
<fail message="FE build FAILED."/>
</then>
</if>
</then>
<else>
<echo message="Async FE build was not started."/>
</else>
</if>
</sequential>
</macrodef>
|
It begins by checking if the fe.build.started property is set, indicating that the FE build has started. If the build has started, it proceeds to wait for the build to finish.
During the waiting period, it continually checks if the fe.build.ended property is set, indicating the completion of the FE build. If the build ends within a specified time frame (up to 15 minutes, specified by the fe.build.async.wait.min property), it indicates the completion of the build process. However, if the build does not finish within that timeframe, it fails the build process.
After waiting for the FE build to complete, the macrodef checks the exit codes of both the FE build (fe.build.exit.code) and npm install (fe.install.exit.code) processes. If either of these processes fails (i.e., the exit code is not 0), it prints an error message and fails the entire build process.
Updating existing build process to use async FE build
To begin with, separate targets will be introduced to execute BE and FE builds using ant extension points, simplifying integration with various custom build process implementations across different projects.
1
2
3
4
5
6
7
8
9
10
|
<target name="be" depends="clean,all,build_be" description="Build BE only">
</target>
<target name="fe" depends="build_fe" description="Build FE only">
<wait_fe_build_async/>
</target>
<extension-point name="build_fe"/>
<extension-point name="build_be"/>
|
Following this, a project-specific target for the build_fe extension point will be created to initiate the FE build.
1
2
3
4
5
6
|
<target name="build_fe_blog" extensionOf="build_fe">
<sequential>
<build_fe_common_async feBaseDir="${HYBRIS_BIN_DIR}/custom/blogstorefront/web"/>
</sequential>
</target>
|
These changes introduce the ant fe target, which will start the FE build and wait for its completion. Next, existing build process should be updated, so ant clean all also starts and waits for async FE build execution.
If the FE build in the project is fully independent, the execution of npm tasks can be removed from the current location(usually it is at the end of storefront_after_build macrodef or inside storefront_compileuisrc_executor macrodef), and the FE async build execution can be initiated after clean but before build in ant clean all. If the FE build still depends on the BE build, the execution of npm tasks can be replaced with build_fe_common_async (or build_fe_blog), and efforts should be made to make the FE build independent (see Common FE dependency on BE section).
To trigger the FE build after clean but before build, the following macrodef can be used:
1
2
3
4
5
6
7
8
9
10
11
12
|
<macrodef name="_after_clean">
<sequential>
<!-- Start FE build async on "ant clean all" only -->
<if>
<equals arg1="${ant.project.invoked-targets}" arg2="clean,all"/>
<then>
<runtarget target="build_fe_blog"/>
</then>
</if>
</sequential>
</macrodef>
|
A generic _after_clean is used instead of the extension-specific storefront_after_clean as OOTB SAP Commerce doesn’t execute the after_clean macrodef for storefront extensions. Note that we initiate the execution of an ant target, which is a project-specific definition of the build_fe extension point.
To wait for the completion of the FE build, a similar approach is employed after the execution of the build macrodef:
1
2
3
4
5
6
|
<macrodef name="_after_build">
<sequential>
<wait_fe_build_async/>
</sequential>
</macrodef>
|
Conclusion
Implementing parallel execution of FE and BE builds using Ant tasks offers a practical solution to enhance the build process of SAP Commerce projects. By optimizing build times and decoupling FE and BE builds, developers can streamline the development workflow and improve overall efficiency. Embracing parallelization is key to unlocking the full potential of your development pipeline and accelerating time-to-market for your online storefronts.
Appendix
How to resolve addons dependencies of FE build on BE build
Usually the only dependency of FE on BE are addons, where BE copies LESS and JS files. This dependency can be easily resolved by having a custom addons.less file with imports for required addons, for example:
1
2
3
4
5
6
7
|
@import "../../../../addons/assistedserviceyprofileaddon/responsive/less/assistedserviceyprofileaddon.less";
@import "../../../../addons/assistedservicepromotionaddon/responsive/less/assistedservicepromotionaddon.less";
@import "../../../../addons/textfieldconfiguratortemplateaddon/responsive/less/textfieldconfiguratortemplateaddon.less";
@import "../../../../addons/orderselfserviceaddon/responsive/less/orderselfserviceaddon.less";
@import "../../../../addons/b2bacceleratoraddon/responsive/less/b2bacceleratoraddon.less";
@import "../../../../addons/commerceorgaddon/responsive/less/commerceorgaddon.less";
@import "../../../../addons/captchaaddon/responsive/less/captchaaddon.less";
|
JS files, usually are handled by npm/grunt build itself.
Separate npmancillary-npm.xml file with OOTB npm commands
Could be used separate npmancillary-npm.xml file, which should be imported in extension buildcallbacks.xml with <import file="npmancillary-npm.xml"/>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
|
<project>
<!-- copied from npmancillary/buildcallbacks.xml, target 'repairnpm'; converted to a macrodef -->
<macrodef name="repairNpmMacro">
<sequential>
<echo message="fixing node/npm symlinks and permissions"/>
<detectOS/>
<property
file="${ext.npmancillary.path}${file.separator}resources${file.separator}ant${file.separator}${os.family}.properties"/>
<if>
<not>
<os family="windows"/>
</not>
<then>
<!-- sometimes the npm link is converted to file -->
<echo message="OS family - ${os.family}, execute file: ${ext.npmancillary.path}/resources/npm/repairnpm.sh"/>
<exec executable="${ext.npmancillary.path}/resources/npm/repairnpm.sh">
<arg line="${os.family}"/>
</exec>
</then>
</if>
</sequential>
</macrodef>
<macrodef name="invokeNpmInCustomDir">
<!-- This macro is compatible with CCv2 -->
<attribute name="workingDir" default=""/>
<attribute name="arguments" default=""/>
<attribute name="resultproperty" default="npm.run.result.property"/>
<sequential>
<echo message="Running [npm @{arguments}] in directory @{workingDir}"/>
<property environment="env"/>
<detectOS/>
<property
file="${ext.npmancillary.path}${file.separator}resources${file.separator}ant${file.separator}${os.family}.properties"/>
<if>
<os family="windows"/>
<then>
<exec executable="cmd" dir="@{workingDir}" failonerror="true" resultproperty="@{resultproperty}">
<arg value="/c"/>
<arg value="${loc.NODE_HOME}${file.separator}npm"/>
<arg line="@{arguments}"/>
</exec>
</then>
<else>
<exec dir="@{workingDir}" executable="${loc.NODE_HOME}${file.separator}bin${file.separator}npm"
failonerror="true" resultproperty="@{resultproperty}">
<env key="PATH" value="${loc.EXTRA_PATH}${path.separator}${env.PATH}"/>
<arg line="@{arguments}"/>
</exec>
</else>
</if>
</sequential>
</macrodef>
<macrodef name="invokeNodeInCustomDir">
<!-- This macro is compatible with CCv2 -->
<attribute name="workingDir" default=""/>
<attribute name="arguments" default=""/>
<sequential>
<echo message="Running [node @{arguments}]"/>
<property environment="env"/>
<detectOS/>
<property
file="${ext.npmancillary.path}${file.separator}resources${file.separator}ant${file.separator}${os.family}.properties"/>
<if>
<os family="windows"/>
<then>
<exec executable="cmd" dir="@{workingDir}" failonerror="true">
<arg value="/c"/>
<arg value="${loc.NODE_HOME}${file.separator}node"/>
<arg line="@{arguments}"/>
</exec>
</then>
<else>
<exec dir="@{workingDir}" executable="${loc.NODE_HOME}${file.separator}bin${file.separator}node"
failonerror="true">
<env key="PATH" value="${loc.EXTRA_PATH}${path.separator}${env.PATH}"/>
<arg line="@{arguments}"/>
</exec>
</else>
</if>
</sequential>
</macrodef>
</project>
|