29/06/22

Handling big JSON files in iOS

By Oleg Shanyuk

Let me talk you through a story of single file optimisation with a good impact on application size and performance.

It is important to mention that changes in the fundamental components affect an app a lot, and you might require a lot of people to introduce these changes safely. Therefore, a good approach is to recognise this impact, and check if it is possible to avoid the breaking changes while working towards the desired outcome.

The Problem

One of the metrics we track at Delivery Hero is application size. To keep the apps in “good shape”, we periodically revisit and look into the biggest files, and one of ours was Translations.json.

Parsing this file takes place during each app start, and from some dozens of milliseconds initially, this now grew to above half a second! (that’s a lot of time during app launch).

An Interesting fact about this file is that it’s growth was constantly adding up as the application developed, which is connected to the amount of countries we operate in, and the amount of services we offer via the apps.

Why use JSON?

Why use JSON for translations in the iOS apps? Let me walk you through it.

Of course you can argue that it’s still possible to have optimization in place with native strings, but we must also think of the developer experience. Below is how it is with GitHub out of the box. As you can see, identifying what has changed in the translations is a problem.

Trick is .strings are not “diffable” by default. 

Below is how diffs looks with JSON “out of the box”:

This tiny detail makes it way more reliable to use for collaborative work on GitHub, and having local diffs quickly. Of course, this problem can be solved, but we would have to introduce custom setups for git and strings, and do it on multiple levels. 

Let’s focus for now on what we have, and take a look into the numbers.

The Measurements

Before going deeper, let’s take the initial metrics using the same device (iPhone XR) before and after the optimisations. Also, let’s measure JSON approach vs native iOS strings to store localisation texts.

What was measured:

  • the size of the Translations (JSON)
  • the parsing time, taken during the app start

We’ve taken the ‘worst’ cases – the largest/slowest numbers.

According to these measurements it’s easy to conclude:

  • JSON split is definitely helpful in gaining a lot of time during the app start phase
  • native strings can narrowly win in performance in some cases (but carry other problems)
  • the amount of translation keys is a big factor, and it seems
  • like it impacts .strings more, rather than JSON

The Solution

Cutting to the chase, I’d like to share what we’ve achieved:

  • We split JSON into multiple files, cutting initial parse time dramatically (over 20x)
  • We applied ‘smart diffs’ by eliminating duplicates in json files, and were able to cut total file size by more than 50%

Here’s how we moved towards the solution:

We decided to start with the application launch time impact

Splitting a single file into multiple (based on country and locale) was a good step towards optimizing the app start time, impacting the UX directly. This step alone saved us 500ms of time.

The second step is to reduce the amount of translations

The old translations structure included all the translation keys in all possible variations of the countries and languages. As we are using English as a base locale, it was worth checking how many strings are not translated yet, and therefore duplicating across many translations. We were able to reduce the amount of translation content from around 13-14Mb to 5.5Mb! That, of course, led to some modifications in the app translations engine, but it was totally possible to keep all changes ‘under the hood’, and would not require any modifications in the public interface of our translations framework.

Interesting effect of the second step of optimisation:

  • Most of the translation files are parsed in only 10ms
  • EN translation (the basic one) is parsed under 20ms
  • Average FULL loading time ( EN + target translation is 25ms)

What we compared:

  • the performance and size impact of the JSON format for storing strings to the native .strings format used in iOS
  • single JSON file parsing time to the multiple JSON files (one per language, similar to what we have with .strings)

The Results

Therefore, we slashed more than half of the translation size, and got a significant app start time impact, improving the parsing step by 25x times!

What if we used .strings?

To ensure we found a good solution, let’s compare our optimized JSON files to the ‘native’ strings solution 

How we did it: 

  1. Exported translations on a per-language basis into ‘native’ .strings format files.
  2. Ran a simple code to load translations from the bundle, and measured it.

Results of the solution:

Therefore, we are running at least as good as the system default.

Interestingly, for “.strings” the parse time was 0.025s for pretty much all locales – I think it depends on the number of translation keys more than on the length of the strings.

Conclusions

Quick takes from all of the above:

  • Keep your JSON small – below 500 Kb optimal, and up to 1Mb at worst!
  • Revisit and retrospect your code and solutions – the project might need updates in the well-known and working parts.
  • Monitor your app launch time
  • Find a way to split and minimize your resources, especially required for the app launch
  • Look for opportunities to avoid duplications inside the resources
  • The Developer Experience matters! (That’s why we stick with easy readability vs ‘native’ format)

The Bonus

After implementing the solution and releasing it to the customers, we didn’t stop there. Since we were using JSON, we started to think about how we could optimize the json files further. Use of json-minify formatting looked interesting, but this would make diffs on git look terrible. Therefore, it would make sense to minify translations json files during the build time. So we  did! But instead of only minifying translations, we decided to minify all json files in the application build folder. Of course, we also did it for the release configuration, not affecting the build time for developers. 

Result: we slashed an additional 600Kb+ of the app install size, almost ‘for free’. Many big json files (excluding those dedicated to staying readable by humans) were already minified!

To do so, we applied 2 steps:

  1. created a small json-minify script which accepts only one argument – a full path to the json file
  2. added a build phase for release configuration to run this script for all json files found in the build directory

I can recommend a simple command line open source utility for JSON minification: jq

Our build phase to minify all json files looks this simple. It is designed to run after the Copy Bundle Resources phase – pay attention when you add it in Xcode.

BUILD_DIR_RESOURCES_PATH="${CONFIGURATION_BUILD_DIR}/${UNLOCALIZED_RESOURCES_FOLDER_PATH}"

find "${BUILD_DIR_RESOURCES_PATH}" -type f -name "*.json" -print0 | xargs -0 -n 1 "${SRCROOT}/scripts/json-minify.sh" &>/dev/null

and this is our json-minify.sh script: 

#!/usr/bin/env bash

# fail if any command fails
set -e
file=$1
tempFile=$(mktemp /tmp/json.XXXXXXXXX)

echo "🚮 temp file: $tempFile"
echo "🗜 optimizing: $file"
    
jq -c . < “$file” > “$tempFile” && mv -f “$tempFile” “$file”

echo "✅ $file minified"

Add this to your app today, and benefit from minifying json files!


Thank you, Oleg, for sharing your learnings with us! 

Do you enjoy looking into apps and finding good optimization solutions? Check out some of our latest openings, or join our Talent Community to stay up to date with what’s going on at Delivery Hero and receive customized job alerts!