Javascript performance testing

I've been horribly disappointed with IE's performance with some of my javascript, so the point that I've pretty much decided to simply hide the troublesome functions from IE users. That's nearly 85% of our audience who won't get to experience the work I've put in to some of this stuff. I thought I needed to get some actual figures on performance, so I took the worst offender and turned it into a test case. Then I thought while I'm doing that I might as well compare jQuery 1.2.6 to jQuery 1.3.1. So I made a 2nd test case.

First of all, you need to understand that I'm terribly abusing the plugin in question. I'm using treeview to do the online version of our organizational chart. There's a demo showing how it works with large lists. That demo has 290 total list items, 53 of which are expandable. Our org chart has 403 list items, 81 of which are expandable. It's large enough that I had to edit the default images to extend all the way down the fully expanded list.

I added a crude counter to the plugin. Here's the code. The lines involving Date() and getTime() and alert() functions are my additions (114 to 127 or so). It's far from perfect, but it should be equally imperfect for all browsers and therefore free of bias. What this does is whenever the expand all or collapse all function is triggered, it grabs the time early on in that process, then grabs the time again near the end of that process, computes the difference (in milliseconds) and alerts the value. In each browser I expanded the list, recorded the value, collapsed the list, recorded that value, and repeated the process until I had 15 measurements for both the expand and the collapse feature.

The Data

We'll do the Mac browsers first. I'm running on a 20 inch iMac with a 2.0 gHz Intel Core Duo processor and 4 gigs of RAM. I ran these tests under "normal" conditions. So I had other applications going and in the case of Firefox several other tabs open.

All figures are in milliseconds.

Firefox 3.0.6

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	838 840 847 826 827 821 839 835 827 827 825 839 824 816 845	1178 1184 1183 1188 1188 1184 1187 1205 1186 1190 1189 1182 1185 1182 1190	506 514 511 493 482 513 480 510 493 484 508 484 492 512 491	1006 1024 999 998 1011 998 1010 1003 1009 1008 1005 1002 1000 997 994
Min	816	1178	480	994
Max	847	1205	514	1024
Range	31	27	34	30
Median	827	1186	493	1003
Average	831.73	1186.73	498.2	1004.26

The most surprising thing here is that jQuery 1.2.6 is actually a bit faster than 1.3.1. Of course, 1.3.2 is supposed to release soon. And as we'll see that's not true for every browser. Collapsing takes longer than expanding. I'll leave explaining that to someone that knows more about jQuery DOM transversal.

Safari 3.2.1

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	262 256 267 269 264 269 266 267 270 270 267 269 292 267 267	161 159 164 163 167 164 169 165 163 164 165 164 164 166 166	311 311 326 316 314 311 311 319 312 314 310 316 318 318 317	147 146 149 151 148 148 146 149 149 149 149 151 148 152 150
Min	256	159	310	146
Max	292	169	326	152
Range	36	10	16	6
Median	267	164	314	149
Average	268.13	164.27	314.9	148.8

At first I thought the performance boost for 1.3.1 here was pretty small, but I think the fact that all the numbers involved is tiny threw me off. If you add up the median figures, you'll find a difference of about 7.5%. That's nothing compared to the relative advantage Safari already has over most of the other browsers, but I think it's still a decent speed boost for the new jQuery release.

Safri's js engine is awesome. :)

Can anyone explain why in Safari collapse is quicker than expand, but in Firefox it's the other way around?

Opera 9.63

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	692 690 737 667 711 703 715 703 704 692 719 680 731 695 682	698 709 721 709 715 730 697 695 692 696 705 704 697 713 696	875 944 904 875 872 869 864 929 905 910 891 885 926 857 918	691 711 692 692 684 692 741 688 652 687 674 707 701 692 690
Min	667	692	857	652
Max	737	730	944	741
Range	70	38	87	89
Median	703	698	891	692
Average	701.4	705.13	894.93	692.93

Opera seems slow compared to Safari, but it's on par with Firefox, and more than tolerable. We also see a decent speed boost with 1.3.1 here.

Windows Browsers

I'm running Windows XP via virtualization with 412 megs of RAM. These tests were run with no other running applications, so more "ideal" conditions than the Mac browsers got. Then again, the Macs get a ton more RAM and aren't running in a virtual machine. I tested both Firefox 2 and Firefox 3 because a decent portion of our Firefox using audience haven't upgraded yet.

Firefox 2.0.0.20

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	1018 1069 1019 1019 1019 1019 989 1049 1019 989 1029 1009 1029 999 969	1169 1109 1129 1129 1139 1159 1129 1189 1149 1129 1128 1129 1129 1189 1119	629 629 609 649 609 639 619 629 619 649 630 629 640 629 639	740 719 739 729 749 749 749 730 739 739 759 750 759 719 769
Min
Max
Range
Median
Average

Here we're starting to get slower than I'd like. A full second for anything to happen is a bit much. Then again, 403 list items in 81 lists is a lot of munch on. Once again we loose a little speed with the new jQuery release. Hmm.

Firefox 3.0.6

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	431 449 461 434 464 472 558 493 481 436 457 442 472 462 458	657 688 648 623 657 708 632 650 695 649 634 649 648 659 625	450 467 424 475 443 456 446 472 460 477 449 464 483 468 440	618 650 634 638 667 637 632 632 646 643 622 614 628 651 640
Min
Max
Range
Median
Average

Here we go! Come on Firefox users, update already!

Safari 3.2.1

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	253 326 277 257 268 288 278 269 244 271 264 281 284 281 293	141 171 148 160 147 143 147 150 151 171 163 158 154 142 147	303 300 324 327 313 312 283 297 355 332 367 332 360 317 307	130 133 129 134 135 134 129 130 132 130 133 135 148 133 129
Min
Max
Range
Median
Average

Safari continues to be awesome, even on Windows. :)

Opera 9.63

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	470 480 469 480 480 499 490 580 469 519 460 470 479 563 480	399 410 429 409 419 399 419 409 400 400 400 430 410 399 399	949 989 949 929 939 939 939 929 939 939 959 949 958 1009 989	370 380 390 409 400 390 370 390 380 400 399 420 380 400 420
Min
Max
Range
Median
Average

With Opera on Windows, we see some pretty solid speed boosts with the 1.3.1 release of jQuery.

Chrome 1.0.154.48

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	229 203 191 195 158 208 201 188 204 244 360 327 307 305 178	88 77 119 88 79 106 84 86 95 158 116 145 135 61 81	512 598 546 607 607 640 526 674 545 631 549 635 581 616 531	111 112 133 115 109 105 103 103 105 106 133 112 109 110 114
Min
Max
Range
Median
Average

Chrome also sees some good boosts from 1.3.1 and manages to out awesome even Safari. When you look at the range as a percentage, it's all over the place. Sometimes the alert box would have a check box in it that would say something to the effect of "prevent this site from producing alert boxes" and I think those tended to take longer to generate. Maybe Chrome's doing other stuff behind the scenes. I'd need to do more specific research to figure this one out.

IE 7.0.5730.13

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	2990 2897 2867 2917 3016 2877 2877 2847 2907 2897 2877 2916 2897 2888 2848	1718 1738 1832 1718 1668 1708 1678 1678 1698 1709 1728 1738 1689 1688 1708	2538 2587 2557 2557 2577 2568 2567 2557 2548 2707 2548 2708 2558 2568 2568	1248 1219 1249 1248 1279 1249 1208 1228 1258 1229 1229 1239 1389 1229 1239
Min
Max
Range
Median
Average

Ugh! I'm in browser hell! What's up with this? 2.5 to 3 seconds to expand the list? Come on!

IE6

I don't know the exact version number here because of the way the Multiple IE installer works.

jQuery:	1.3.1		1.2.6
	Expand	Collapse	Expand	Collapse
Tests	10779 12577 13060 14085 15034 16123 17272 19549 19859 21121 23080 23249 24961 24770 25953	4935 5165 5774 5964 6314 7032 7152 8021 8231 8611 9082 9402 9892 9959 10349	11078 12617 13167 14785 15684 16876 19643 21740 20642 21947 22726 23925 36235 28810 29422	3899 4005 4356 4715 5105 5534 6453 7366 6842 7453 7801 8241 9700 9364 9849
Min
Max
Range
Median
Average

No. I was wrong before. This is browser hell. Notice how each test gains about 1 full second? I think that means we've got memory leaks. I closed and relaunched the browser between test cases to avoid skewing the results between jQuery versions.

I'll complete the number crunching and do a bit more in depth analysis tomorrow. It's 5 o'clock and I'm heading home for the day.