metagoto's blog

UPDATE (2010-01-09T22....Z): a small change (yet to be properly benchmarked) yields linear performance gains: the extension brings about 30% more RPS than the user code in all tests. /UPDATE

Ok, the C implementation of SplClassLoader is here on github. I am afraid I won't have much time in the coming days to follow what is going on. Anyway I took the time to do some benchmarks with ApacheBench. I will present the results in this post.

In order to ease things, I wrote a python script that generates a dummy directories subtree filed with php files containing blank classes with their proper namespace. The script also generates a php file named batch_instances.php (by default) which is responsible for instantiating all the generated classes in a row. It looks like that:

$a = new \vendor0\AA;
$a = new \vendor0\AAA;
$a = new \vendor0\B_B;
$a = new \vendor0\B_BB;
$a = new \vendor0\sub0\AA;
$a = new \vendor0\sub0\AAA;
...
$a = new \vendor1\sub0\AA;
$a = new \vendor1\sub0\AAA;
...
$a = new \vendor1\sub1\sub1\AA;
$a = new \vendor1\sub1\sub1\AAA;
$a = new \vendor1\sub1\sub1\B_B;
$a = new \vendor1\sub1\sub1\B_BB;

That php file is then included for the various benchmarks. The python script (fs_gen.py) is located under tests/for_bench but it is not integrated with the extension tests suite.

Setup

• MacBook Pro 2.53ghz 4gb ram
• nginx 0.8.27
• PHP_5_3_FPM trunk compiled with --disable-all --enable-maintainer-zts. Unix sockets.

Note: APC was not loaded (nor any other accelerator.. I would have to install it btw).

Protocol

I did 3 kind of tests:

• SplClassLoader() used without namespaces nor path arguments. We have to configure the include_path.
• SplClassLoader('vendorN') where N goes from 0 to 3 at maximum. It means we have several 'vendor' namespaces. Note that the second argument (the path) is not passed to the constructor.
• SplClassLoader('vendorN','/pathN') where the path is absolute. The php include_path is not used.

For example, the second test bootstrap file looks like this:

<?php
// 'vendorN' directories are under lib/
set_include_path(
   __DIR__.PATH_SEPARATOR
  .__DIR__.DIRECTORY_SEPARATOR.'lib'.PATH_SEPARATOR
  .get_include_path() // mine is empty btw
);

$v0 = new SplClassLoader('vendor0');
$v0->register();

$v1 = new SplClassLoader('vendor1');
$v1->register();

include 'batch_instances.php';

Since both implementations declare the same class SplClassLoader be it a user script or an extension, the php implementation has been renamed to SplClassLoaderUser.

I noted each result returned by ApacheBench. For each test, a few warm ups then 10 runs in a row. Results in the following tables are in requests per second. Also, the lowest and the highest numbers were removed prior calculating the mean.

The exact arguments passed to ab are not known at that time ;) Depending on the number of classes loaded, sometimes it was -n 3000 or -n 1000 but always -c 1 and of course always the same for a given test.

Results

24 classes under a unique 'vendor0' namespace containing several sub namespaces:

        test 1     test 2      test 3
        simple     with ns     with ns+path

user    496        429         449
ext     554        543         576
ratio   1.117      1.266       1.282

70 classes under two namespaces containing several sub namespaces:

        test 1     test 2      test 3
        simple     with ns     with ns+path

user    221        182         192
ext     232        228         246
ratio   1.050      1.252       1.281

Note: there were no underscore characters in the classes names used for the above bench.

126 classes under two namespaces containing several sub namespaces:

        test 1     test 2      test 3
        simple     with ns     with ns+path

user    119         99         104
ext     126        124         133
ratio   1.059      1.253       1.279

Conclusions

Results are indeed extremely regular. In fact I performed several other tests with different directories layouts but ratios were roughly the sames.

Internally, the C implementation does nothing more than the php implementation. spl_autoload_register() is called through the Zend Engine. There is no magic door. When more computations are involved as in the test number 3 (comparing namespaces and appending paths) the C implementation is ahead.

In fact these numbers suggest that we better have to pass both a namespace and an absolute path to the SplClassLoader constructor when using the extension. On the contrary the php implementation performs better with no argument at all, ie. no namespace comparison. Low level string manipulations are faster in C. All in all, both implementations call include() through the ZE.

One advantage of course concerning the extension is that the class is registered at module initialization (MINIT). The class remains in the engine between requests (if the sapi allows it).

PS. Why did I write that extension? I was looking for a simple (no dependencies) extension to illustrate a tutorial I am supposed to write (second part) for this blog (in french). I came up with SplClassLoader but I am by no way a ZE specialist.