Strange Launch time test results in Windows Phone Marketplace Test Kit

With Windows Phone 7.1 (Mango) SDK Microsoft announced Windows Phone Marketplace Test Kit, which provides a suite of automated, monitored, and manual tests to help prepare your applications to be accepted in the Marketplace the first time you submit them. The test kit enables you to identify and fix issues prior to Marketplace submission, which will save your time in the submission process. The test kit is integrated within Visual Studio and you can open it by choosing the Open Marketplace Test Kit option in Project Menu. This test kit is a must for anyone trying to certify their app, because it really helps you in preparing your application for publishing on Marketplace.
The test kit contains the follow test categories:

• Application Details
• Automated Tests
• Monitored Tests
• Manual Tests

So, when I started with testing my application everything went smoothly; I prepared my screenshots, all automated test passed successfully. But when I run Monitored tests and Launch test failed, stating that my app took 5.9 seconds to start. Performance and Resource Management requirement: 5.2.1. Launch time (http://msdn.microsoft.com/en-us/library/hh184840(v=VS.92).aspx) states following:
“The application must render the first screen within 5 seconds after launch.”

This was very strange for me since I thought that I did good code optimization work and my app in reality starts on device almost immediately (0.5 seconds). I started to investigate what caused this behavior, but with no improvement, even when I commented all code in my application. My application is simple pivot app with dozen of standard WP7 controls and on Bing Map control.
So I started from beginning with testing blank Windows Phone Application. Launch time result was 1.8 seconds. Then I tested blank Windows Phone Pivot Application, and result was 2.6 seconds. When I added Map control to pivot item Launch time was 4.5 seconds! Almost 5 seconds and I didn’t add any other assembly, control, line of code or anything else.
So this is problem. No matter that all these apps start immediately, for some reason Test Kit reports much higher numbers. Also, seems that Map control causes lot of work for UI thread during parsing and creation of Bing Map for Silverlight object from XAML and drawing it’s visuals during initialization. Microsoft has good article on MSDN regarding Performance Considerations in Applications for Windows Phone: http://msdn.microsoft.com/en-us/library/ff967560(v=VS.92).aspx. One of suggestions is to use Background Threads to avoid complex processes that can block the UI Thread (see article: How to use a Background Worker). So I decided to put background worker in page initialization and add Map dynamically in RunWorkerCompleted event:

        public MainPage()
{
InitializeComponent();

BackgroundWorker backroungWorker = new BackgroundWorker();
backroungWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(backroungWorker_RunWorkerCompleted);
backroungWorker.RunWorkerAsync();
}

private void backroungWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
Map map2 = new Map();
this.ContentPanel.Children.Add(map2);
}

Or for Pivot app:

private void backroungWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
Map map2 = new Map();
this.PivotItemThree.Content = map2;
}

Launch test results were 1 second shorter than implementation in classic way by adding XAML code to your page. Here are results for all app types:

Application type	Launch time results
Windows Phone Application	[INFORMATION] : Application took 1.8 seconds to launch.
Windows Phone Pivot Application	[INFORMATION] : Application took 2.6 seconds to launch.
Windows Phone Application with Map	[INFORMATION] : Application took 4 seconds to launch.
Windows Phone Pivot Application with Map	[WARNING] : Application took 4.5 seconds to start. This is close to the allowed limit for launch time being 5 seconds. For more information, see the Application Certification Requirements for Windows Phone at http://go.microsoft.com/fwlink/?LinkId=223631.
Windows Phone Application with Map created on RunWorkerCompleted event	[INFORMATION] : Application took 3 seconds to launch.
Windows Phone Pivot Application with Map on RunWorkerCompleted event	[INFORMATION] : Application took 3.5 seconds to launch.

If you have issues of this type consider using of dynamically adding controls to your main page on Background Thread. I am not sure is this behavior of Test Kit same in official Marketplace submission process, since real app response time and time from test results are really different. Also, I conducted these tests only on my machine and my device (LG E900 Optimus), so I am not sure is this common behavior in all development environments. Please leave comment with your results on Launch time tests.