React Native Performance Analysis of Hermes on iOS

React Native Performance Analysis of Hermes on iOS

Introduction

This document contains a performance analysis on the impact of enabling the Hermes engine on iOS. We analyze impact on cold start, memory and app size using a local device. A similar document for Android is available here: RN Bundle Load Impact on Cold Starts [Android/Hermes].

Background

Hermes is an open-source JavaScript engine optimized for React Native. According to the official documentation, using Hermes should result in improved start-up times, decreased memory usage and smaller app size when compared to JavaScriptCore engine. We enabled Hermes on Android, with Hermes bytecode bundle, in October 2023 (see RN Bundle Load Impact on Cold Starts [Android/Hermes]) and saw significant performance improvements (app start time down from ~5s to ~1s). Hermes engine uses AOT, as opposed to the default JavascriptCore engine which uses JIT, so that allows to parse and compile JS code at build time, hence improving things such as startup times.

Integration

Hermes was introduced as part of React Native v0.64, so it’s currently available in both our Greenfield and Brownfield codebases (v0.72.6). Greenfield already has Hermes enabled, while on Brownfield it had to be turned on via the hermes_enabled flag in the CocoaPods Podfiles of AlcatrazReactNative-Debug and AlcatrazReactNative-Release. Along this change we also had to instruct to build the libReact-hermes.a instead of the libReact-jsc.a previously utilized in the JavascriptCore engine. Subsequent changes to the upstream packages AlcatrazNativeCore and IndigoAlcatraz were also necessary to ensure they would build against the Hermes dynamic framework and static library. Main Brownfield change:

A corresponding change in DragonFly was also necessary in order to build the RN Bundle with Hermes bytecode, as opposed to the default ASCII one, which allows to benefit from the full Hermes optimizations:

Performance Analysis

In order to measure the performance differences using Hermes we utilized the following:

  • Used a physical device (iPhone 12) running iOS 17.3.1

  • Built a Release build locally using latest mainline build (based off 24.4.0)

  • Used the same build with Hermes enabled and disabled (without any extra changes)

  • Ran few iterations for each scenario (at least 5x for Cold Starts and a 3-4x traces for memory profiling)

Note: It’s understood that this is just an approximation given we will be using a single device in pretty ideal conditions and limited sampling. We will need to get latency data from customers on the field in order to get a more conclusive representation of Hermes’ impact.

Cold Start

Methodology: Cold started the app into Find Landing Page for 5x iterations locally. Find Landing page was chosen since it’s already migrated to React Native and it’s a landing page reachable from a cold start.

Our “Cold Start time” definition is the time from customer invoking the app until the landing page is fully rendered. See Cold Start Piecemeal Latency metrics Proposal. However we haven’t yet updated our Cold Start Native telemetry to include the pageLoad time of a React Native page (initial CR and parent JIRA), so we cannot rely on telemetry alone to measure the impact of these changes. Hence, for this performance analysis, we ended up looking at the following:

  1. Native to React Native latency: Timestamp difference between two log statements added locally where (1) we initialize the RN bridge (from ReactNativeViewController initializeBridge method) and (2) the first log emitted from RN side (from App.tsx in DragonFly). This should essentially measure the RN bootstrap/bundle load time.

  2. React Native latency: pageLoad (latencyInfo) event emitted from Find Landing Page:

    1. totalDurationMs of the pageLoad: this measures overall what’s the impact on the total pageLoad time

    2. start leg, which is a pieceDuration of the totalDurationMs: this measures specifically the time between Native to React Native

The raw data is available at Appendix A: Start App Times here is the summary table:

Hermes disabled

Hermes Enabled

Delta

1) Native to RN latency (RN bootstrap time)

1588ms

690ms

2.3x

2a) RN latency: totalDurationMs of FLP

439.65ms

275.16ms

1.6x

2b) RN latency: start leg of FLP

163.5ms

46.24ms

3.5x

Overall this indicates a pretty strong improvement across all these dimensions when using Hermes engine, as also demonstrated by this side-by-side video comparison Appendix B: Comparison Video

Memory

Methodology: cold-started the app into Find Landing Page and let the page rest for ~1 minute, then looked at the total memory allocation. Repeated this a few times to account for memory variation. We followed the iOS memory profiling described on Amazon Music iOS Memory Footprint Profiling.

Hermes enabled: Hermes enabled

Hermes disabled: Hermes disabled

Overall the memory comparison seems to indicate not much of a difference between the Hermes enabled (421MB) vs Hermes disabled (419MB) scenarios, even though theoretically Hermes promises better memory management. The tests were run for a few iterations (3-4x) and the results were pretty close with each other, with some understandable variation due to the unpredictable nature of the memory allocation and garbage collection. Overall the difference seems negligible and more sophisticated testing should be performed (will engage with Turbo team).

App Size

Methodology: Measured the IPA size via the App Size tool integrated in the Alcatraz-mainline pipeline. This does not however measure the actual download size customers will see, as it would require uploading it to testflight (which is expected to decrease)

Using the App Size tool we noticed around 2MB app size increase with Hermes engine enabled, and according to the this article this should be expected since Hermes libraries need to be shipped with the app, unlike the default JavascriptCore version of the engine which comes built-in with every iOS version.

Summary

Enabling Hermes changes seems to provide beneficial improvements mostly to pageLoad times (0.5-1s improvement), see Appendix B: Comparison Video. In terms of memory consumption didn’t seem to be much affected while there’s a small app size increase (+2MB) due to the Hermes engine being shipped separately, which should be an acceptable trade-off given the significant app start improvements. We will gather additional data from Turbo team and from customers for more conclusive data.

Next Steps

  • Get latency data from Turbo team (pageLoad and memory)

  • Get latency data from customers in the field, for this we can monitor the FLP latencyInfo data https://tiny.amazon.com/tm73wwrc/iGraph

  • Verify actual app size impact from Test Flight

Glossary

  • React Native (RN): React Native framework for building native applications using JavaScript and React

  • Hermes: Hermes is an open source JavaScript engine designed to optimize performance by reducing app launch time and precompiling JavaScript into efficient bytecode.

  • JavascriptCore (JSC): Default JavaScript engine developed by Apple and utilized in various Apple platforms (iOS, macOS, watchOS)

  • RN Bundle: Compiled JavaScript code that is bundled with the application

  • AOT (Ahead-of-Time) Compilation: A technique where code is compiled into machine language before execution, improving performance by reducing runtime overhead.

  • ** **** **** JIT (Just-in-Time) Compilation**: A technique where code is compiled during runtime, allowing for optimizations based on runtime data.

  • Bridge: Mechanism for communication between JavaScript code and native code in React Native

Resources

Appendix A: Start App Times

Following timings were gathered on a local Release build using an IPhone 12 (iOS version 17.0.2) with 5x iterations. It is understood this is only a limited sample but should provide initial directional data.

Hermes Disabled

    1. Native to RN latency (RN bootstrap time): 1.5s

  • 2a) RN latency: totalDurationMs of FLP: 363.42ms

  • 2b) RN latency: start leg of FLP: 116.84ms

13:39:49.4116 [<NSThread: 0x283822200>{number = 3, name = (null)}]  ReactNativeViewController ALCE initializeBridge
13:39:50.9146 [<NSThread: 0x283803c00>{number = 10, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = false
13:39:53.1843 [<NSThread: 0x28377e500>{number = 21, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x283824d40>: ALCE latencyInfo attributes: ["latencyOperation": Optional("pageLoad"), "latencySubtype": Optional("findV3"), "durationMilliseconds": Optional("363.4291259981401"), "pieceDurations": Optional("[{\"durationMilliseconds\":\"116.84312499998487\",\"latencyPiece\":\"start\"},{\"latencyPiece\":\"init\",\"durationMilliseconds\":\"0.5086250000167638\"},{\"latencyPiece\":\"request\",\"durationMilliseconds\":\"0.09233399998629466\"},{\"latencyPiece\":\"response\",\"durationMilliseconds\":\"112.85687499999767\"},{\"durationMilliseconds\":\"45.66381699973135\",\"latencyPiece\":\"processing\"},{\"latencyPiece\":\"rendering\",\"durationMilliseconds\":\"87.46434999842313\"}]"), "networkType": nil, "latencyCode": Optional("pageLoadFindV3")]

    1. Native to RN latency (RN bootstrap time): 1.6s

  • 2a) RN latency: totalDurationMs of FLP: 659.52ms

  • 2b) RN latency: start leg of FLP: 237.52ms

13:45:32.1094 [<NSThread: 0x280c20380>{number = 6, name = (null)}] ReactNativeViewController ALCE initializeBridge
13:45:33.7189 [<NSThread: 0x2803c2180>{number = 25, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = false
13:45:46.2760 [<NSThread: 0x2803a9440>{number = 33, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x2803c1640>: ALCE latencyInfo attributes: ["latencySubtype": Optional("findV3"), "durationMilliseconds": Optional("659.528031994123"), "networkType": nil, "latencyOperation": Optional("pageLoad"), "pieceDurations": Optional("[{\"latencyPiece\":\"start\",\"durationMilliseconds\":\"237.517833000049\"},{\"latencyPiece\":\"init\",\"durationMilliseconds\":\"0.581917000003159\"},{\"latencyPiece\":\"request\",\"durationMilliseconds\":\"0.09945799992419779\"},{\"durationMilliseconds\":\"263.8567500000354\",\"latencyPiece\":\"response\"},{\"latencyPiece\":\"processing\",\"durationMilliseconds\":\"55.16090299654752\"},{\"durationMilliseconds\":\"102.31117099756375\",\"latencyPiece\":\"rendering\"}]"), "latencyCode": Optional("pageLoadFindV3")]

    1. Native to RN latency (RN bootstrap time): 1.73s

  • 2a) RN latency: totalDurationMs of FLP: 466.61ms

  • 2b) RN latency: start leg of FLP: 206.074ms

13:48:25.5241 [<NSThread: 0x283eb2a80>{number = 10, name = (null)}]  ReactNativeViewController ALCE initializeBridge
13:48:27.2582 [<NSThread: 0x2831981c0>{number = 26, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = false
13:48:29.6087 [<NSThread: 0x283184a40>{number = 23, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x283157580>: ALCE latencyInfo attributes: ["pieceDurations": Optional("[{\"durationMilliseconds\":\"206.07474999991246\",\"latencyPiece\":\"start\"},{\"latencyPiece\":\"init\",\"durationMilliseconds\":\"0.5827920000301674\"},{\"durationMilliseconds\":\"0.09066700004041195\",\"latencyPiece\":\"request\"},{\"durationMilliseconds\":\"83.0899999999674\",\"latencyPiece\":\"response\"},{\"durationMilliseconds\":\"47.05741699819919\",\"latencyPiece\":\"processing\"},{\"latencyPiece\":\"rendering\",\"durationMilliseconds\":\"129.70974999456666\"}]"), "durationMilliseconds": Optional("466.6053759927163"), "networkType": nil, "latencyOperation": Optional("pageLoad"), "latencyCode": Optional("pageLoadFindV3"), "latencySubtype": Optional("findV3")]

    1. Native to RN latency (RN bootstrap time): 1.48s

  • 2a) RN latency: totalDurationMs of FLP: 337ms

  • 2b) RN latency: start leg of FLP: 135.92ms

13:50:32.8097 [<NSThread: 0x2815093c0>{number = 8, name = (null)}]  ReactNativeViewController ALCE initializeBridge
13:50:34.2907 [<NSThread: 0x281ad1640>{number = 25, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = false
13:50:36.5265 [<NSThread: 0x281aa9600>{number = 32, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x281ad7b80>: ALCE latencyInfo attributes: ["pieceDurations": Optional("[{\"durationMilliseconds\":\"135.92866600002162\",\"latencyPiece\":\"start\"},{\"durationMilliseconds\":\"0.4268750000046566\",\"latencyPiece\":\"init\"},{\"durationMilliseconds\":\"0.06891699996776879\",\"latencyPiece\":\"request\"},{\"durationMilliseconds\":\"60.8114999999525\",\"latencyPiece\":\"response\"},{\"latencyPiece\":\"processing\",\"durationMilliseconds\":\"36.05664200137835\"},{\"latencyPiece\":\"rendering\",\"durationMilliseconds\":\"103.71048299991526\"}]"), "latencyOperation": Optional("pageLoad"), "latencySubtype": Optional("findV3"), "networkType": nil, "latencyCode": Optional("pageLoadFindV3"), "durationMilliseconds": Optional("337.00308300124016")]

    1. Native to RN latency (RN bootstrap time): 1.63s

  • 2a) RN latency: totalDurationMs of FLP: 371.7ms

  • 2b) RN latency: start leg of FLP: 121.16ms

13:55:07.6470 [<NSThread: 0x281f829c0>{number = 6, name = (null)}] ReactNativeViewController ALCE initializeBridge
13:55:09.2814 [<NSThread: 0x2810f0c40>{number = 26, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = false
13:55:11.5536 [<NSThread: 0x2810f0c40>{number = 26, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x281070a80>: ALCE latencyInfo attributes: ["latencySubtype": Optional("findV3"), "networkType": nil, "latencyOperation": Optional("pageLoad"), "latencyCode": Optional("pageLoadFindV3"), "durationMilliseconds": Optional("371.7000279726926"), "pieceDurations": Optional("[{\"latencyPiece\":\"start\",\"durationMilliseconds\":\"121.1595419999212\"},{\"durationMilliseconds\":\"0.6074580000713468\",\"latencyPiece\":\"init\"},{\"durationMilliseconds\":\"0.1259169999975711\",\"latencyPiece\":\"request\"},{\"durationMilliseconds\":\"62.051833000034094\",\"latencyPiece\":\"response\"},{\"latencyPiece\":\"processing\",\"durationMilliseconds\":\"68.58615198428743\"},{\"latencyPiece\":\"rendering\",\"durationMilliseconds\":\"119.16912598838098\"}]")]

Hermes enabled (with Hermes bytecode)

    1. Native to RN latency (RN bootstrap time): 0.63s

  • 2a) RN latency: totalDurationMs of FLP: 257.41ms

  • 2b) RN latency: start leg of FLP: 49.43ms

2024-02-27 10:52:45.749116-0500 CloudPlayer[1331:403070] [<NSThread: 0x282850a40>{number = 8, name = (null)}]  ReactNativeViewController ALCE initializeBridge
2024-02-27 10:52:46.379990-0500 CloudPlayer[1331:403153] [<NSThread: 0x28273d580>{number = 25, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = true
2024-02-27 10:52:48.512859-0500 CloudPlayer[1331:403092] [<NSThread: 0x28273d580>{number = 25, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x28289de80>: ALCE emitLatencyInfoHelper latencyInfo attributes: ["durationMilliseconds": Optional("257.41121232509613"), "networkType": nil, "latencyCode": Optional("pageLoadFindV3"), "latencyOperation": Optional("pageLoad"), "pieceDurations": Optional("[{\"latencyPiece\":\"start\",\"durationMilliseconds\":\"49.42654198408127\"},{\"durationMilliseconds\":\"0.24283301830291748\",\"latencyPiece\":\"init\"},{\"latencyPiece\":\"request\",\"durationMilliseconds\":\"0.03462499380111694\"},{\"durationMilliseconds\":\"41.08779203891754\",\"latencyPiece\":\"response\"},{\"latencyPiece\":\"processing\",\"durationMilliseconds\":\"31.390148997306824\"},{\"durationMilliseconds\":\"135.22927129268646\",\"latencyPiece\":\"rendering\"}]"), "latencySubtype": Optional("findV3")]

    1. Native to RN latency (RN bootstrap time): 0.702s

  • 2a) RN latency: totalDurationMs of FLP: 232.79ms

  • 2b) RN latency: start leg of FLP: 40.34ms


2024-02-27 10:53:36.374823-0500 CloudPlayer[1335:404326] [<NSThread: 0x280435980>{number = 3, name = (null)}]  ReactNativeViewController ALCE initializeBridge
2024-02-27 10:53:37.076124-0500 CloudPlayer[1335:404412] [<NSThread: 0x280452c80>{number = 10, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = true
2024-02-27 10:53:39.189963-0500 CloudPlayer[1335:404324] [<NSThread: 0x280a1a6c0>{number = 34, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x280b07a80>: ALCE emitLatencyInfoHelper latencyInfo attributes: ["networkType": nil, "latencyOperation": Optional("pageLoad"), "latencySubtype": Optional("findV3"), "latencyCode": Optional("pageLoadFindV3"), "durationMilliseconds": Optional("232.79402512311935"), "pieceDurations": Optional("[{\"durationMilliseconds\":\"40.343291997909546\",\"latencyPiece\":\"start\"},{\"latencyPiece\":\"init\",\"durationMilliseconds\":\"0.25083303451538086\"},{\"latencyPiece\":\"request\",\"durationMilliseconds\":\"0.03545796871185303\"},{\"durationMilliseconds\":\"36.27350002527237\",\"latencyPiece\":\"response\"},{\"durationMilliseconds\":\"30.98184424638748\",\"latencyPiece\":\"processing\"},{\"latencyPiece\":\"rendering\",\"durationMilliseconds\":\"124.90909785032272\"}]")]

    1. Native to RN latency (RN bootstrap time): 0.804s

  • 2a) RN latency: totalDurationMs of FLP: 319.49ms

  • 2b) RN latency: start leg of FLP: 34.03ms

2024-02-27 10:54:05.412038-0500 CloudPlayer[1339:405426] [<NSThread: 0x281bc03c0>{number = 6, name = (null)}]  ReactNativeViewController ALCE initializeBridge
2024-02-27 10:54:06.216511-0500 CloudPlayer[1339:405497] [<NSThread: 0x281be8740>{number = 21, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = true
2024-02-27 10:54:08.360087-0500 CloudPlayer[1339:405506] [<NSThread: 0x281bd7a00>{number = 3, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x28140bb80>: ALCE emitLatencyInfoHelper latencyInfo attributes: ["latencyCode": Optional("pageLoadFindV3"), "networkType": nil, "latencyOperation": Optional("pageLoad"), "latencySubtype": Optional("findV3"), "durationMilliseconds": Optional("319.4866266846657"), "pieceDurations": Optional("[{\"latencyPiece\":\"start\",\"durationMilliseconds\":\"34.03125\"},{\"durationMilliseconds\":\"0.2681669592857361\",\"latencyPiece\":\"init\"},{\"latencyPiece\":\"request\",\"durationMilliseconds\":\"0.03750002384185791\"},{\"latencyPiece\":\"response\",\"durationMilliseconds\":\"44.56912499666214\"},{\"durationMilliseconds\":\"47.58640545606613\",\"latencyPiece\":\"processing\"},{\"durationMilliseconds\":\"192.99417924880981\",\"latencyPiece\":\"rendering\"}]")]

    1. Native to RN latency (RN bootstrap time): 0.711s

  • 2a) RN latency: totalDurationMs of FLP: 249.05ms

  • 2b) RN latency: start leg of FLP: 47.42ms

2024-02-27 10:54:31.366150-0500 CloudPlayer[1343:406502] [<NSThread: 0x280022240>{number = 4, name = (null)}]  ReactNativeViewController ALCE initializeBridge
2024-02-27 10:54:32.077202-0500 CloudPlayer[1343:406583] [<NSThread: 0x280022240>{number = 4, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = true
2024-02-27 10:54:34.199319-0500 CloudPlayer[1343:406600] [<NSThread: 0x280020f80>{number = 3, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x280fe18c0>: ALCE emitLatencyInfoHelper latencyInfo attributes: ["networkType": nil, "pieceDurations": Optional("[{\"latencyPiece\":\"start\",\"durationMilliseconds\":\"47.42083299160004\"},{\"durationMilliseconds\":\"0.2497919797897339\",\"latencyPiece\":\"init\"},{\"durationMilliseconds\":\"0.036625027656555176\",\"latencyPiece\":\"request\"},{\"latencyPiece\":\"response\",\"durationMilliseconds\":\"38.178541004657745\"},{\"durationMilliseconds\":\"27.81993532180786\",\"latencyPiece\":\"processing\"},{\"latencyPiece\":\"rendering\",\"durationMilliseconds\":\"135.3481850028038\"}]"), "latencyCode": Optional("pageLoadFindV3"), "latencyOperation": Optional("pageLoad"), "latencySubtype": Optional("findV3"), "durationMilliseconds": Optional("249.05391132831573")]

    1. Native to RN latency (RN bootstrap time): 0.604s

  • 2a) RN latency: totalDurationMs of FLP: 317.063ms

  • 2b) RN latency: start leg of FLP: 59.97ms

2024-02-27 10:54:58.311704-0500 CloudPlayer[1347:407551] [<NSThread: 0x2815e4e80>{number = 8, name = (null)}]  ReactNativeViewController ALCE initializeBridge
2024-02-27 10:54:58.915368-0500 CloudPlayer[1347:407655] [<NSThread: 0x281a23200>{number = 29, name = (null)}] [CRITICAL] [1.386.0]: ALCE App.tsx isHermes = true
2024-02-27 10:55:01.093165-0500 CloudPlayer[1347:407629] [<NSThread: 0x281bdf540>{number = 36, name = (null)}] [CRITICAL] <MetricsLoggerModule: 0x281a3da00>: ALCE emitLatencyInfoHelper latencyInfo attributes: ["latencySubtype": Optional("findV3"), "durationMilliseconds": Optional("317.0631051659584"), "latencyOperation": Optional("pageLoad"), "networkType": nil, "pieceDurations": Optional("[{\"durationMilliseconds\":\"59.967291951179504\",\"latencyPiece\":\"start\"},{\"durationMilliseconds\":\"0.2911660075187683\",\"latencyPiece\":\"init\"},{\"latencyPiece\":\"request\",\"durationMilliseconds\":\"0.0394589900970459\"},{\"durationMilliseconds\":\"54.90141600370407\",\"latencyPiece\":\"response\"},{\"latencyPiece\":\"processing\",\"durationMilliseconds\":\"49.80451846122742\"},{\"latencyPiece\":\"rendering\",\"durationMilliseconds\":\"152.0592537522316\"}]"), "latencyCode": Optional("pageLoadFindV3")]

Appendix B: Comparison Video

Left side: Hermes disabled Right side: Hermes enabled (with Hermes bytecode bundle)