_app_start performance metric unreliable for iOS 15

[weibel]

Step 1: Describe your environment

• Xcode version: 12.5.1
• Firebase SDK version: 8.9.1
• Installation method: CocoaPods
• Firebase Component: Performance
• Target platform(s): iOS

Step 2: Describe the problem

The _app_start time reported by Firebase Performance varies wildly for all iOS versions >=15. It goes into thousands of seconds for the 95 percentile.

Steps to reproduce:

I only know the numbers from my own project and can not do much to reproduce.

My hypothesis is that this could be connected to the iOS 15 pre-warming, where initializers and other pre-main steps are run preemptively, potentially hours before the app is started and main() is run. My own custom metric is using the technique from https://eisel.me/startup and suffered from the same problem.

[paulb777]

See related discussion at MobileNativeFoundation/discussions#146

[visumickey]

This issue is because of the iOS15 pre-warming all the applications for quick launch. Currently Fireperf measures the app start time to be the time between first object creation until the app is foregrounded. But since pre-warming loads the classes, the start time seem to have happened long before the app comes to foreground which causes the startup time to be way too high.

Currently, we have a limit on the product to ignore start times beyond 60 minutes (as a protection mechanism), but that still seems to be on the higher end making it a concern.

We are ideating on two different approaches:

1. Can we drop app start durations beyond 2 minutes? (Arbitrary 2 minutes duration - but more realistic to believe apps would not be taking more than 2 minutes for startup.
2. Can we ignore the pre-warmup time to believe on life cycle events that are involved only in the app startup? This is tricky since apple does not guarantee any life cycle events that are involved in pre-warming/not. So we are still exploring if this is a possibility.

We will rollout a fix for this real soon once we settle on the right path forward here.

[ryanwilson]

More measurements and details from this blog post as well: https://sourcediving.com/solving-mysterious-logout-issues-on-ios-15-8b818c089466.

1. Can we drop app start durations beyond 2 minutes? (Arbitrary 2 minutes duration - but more realistic to believe apps would not be taking more than 2 minutes for startup.

My understanding is the Watchdog timer would terminate the app for not launching well before the 2 minute mark, so that sounds reasonable to me.

[leotianlizhan]

Hi @weibel , would you be interested in trying out an early access version of the SDK that may fix this issue?

[weibel]

@leotianlizhan Thanks for offering me the opportunity to test the fix. Unfortunately I can not test beta libraries with our project.

[jeremyjiang-dev]

Hi @weibel,
This issue has been fixed in version 8.13.0, which can now detect and exclude pre-warm starts. To see app start metrics for your applications (including iOS 15), please update to the latest Performance Monitoring SDK. All previous versions of the SDK do not measure the application startup time on iOS 15 due to the pre-warming feature introduced by Apple.