Ulf WendelPHP mysqlnd memory optimizations: from 49MB to 2MB (10.4.2014, 18:58 UTC)

Inspired by Antony, Andrey has implemented a memory optimization for the PHP mysqlnd library. Depending on your usage pattern and the actual query, memory used for result sets is less and free’d earlier to be reused by the PHP engine. In other cases, the optimization will consume about the same or even more memory. The additional choice is currently available with mysqli only.

From the network line into your script

Many wheels start spinning when mysqli_query() is called. All the PHP MySQL APIs/extensions (mysqli, PDO_MySQL, mysql) use a client library that handles the networking details and provides a C API to the C extensions. Any recent PHP will default to use the mysqlnd library. The library speaks the MySQL Client Server protocol and handles the communication with the MySQL server. The actions behind a users mysqli_query() are sketched below.

The memory story begins in the C world when mysqlnd fetches query results from MySQL. It ends with passing those results to the PHP.

PHP script mysqli extension/API mysqlnd library MySQL
*.php *.c
mysqli_query()  
  PHP_FUNCTION(mysqli_query)  
  MYSQLND_METHOD(query)  
  simple_command(COM_QUERY)  
  COM_QUERY
  store_result()  
  return result set  

The new memory optimization is for buffered result sets as you get them from mysqli_query() or a sequence of mysqli_real_query(), mysqli_store_result(). With a buffered result set, a client fetches all query results into a local buffer as soon as they become available from MySQL. In most cases, this is the desired behaviour. The network line with MySQL becomes ready for a new command quickly. And, the hard to scale servers is offloaded from the duty to keep all results in memory until a potentially slow client has fetched and released them.

The result buffering happens first at the C level inside the mysqlnd library. The buffer holds zvals. A zval is internal presentation structure for a plain PHP variable. Hence, think of the mysqlnd result buffer as a list of anonymous PHP variables.

PHP script mysqli extension/API mysqlnd library MySQL
*.php *.c
  store_result()  
  Buffer with zvals (MYSQLND_PACKET_ROW),
think: PHP variables
 

The default: reference and copy-on-write

When results are to be fetched from the mysqlnd internal buffers to a PHP script, the default behaviour of mysqlnd is to reference the internal buffer from the PHP script. When code like $rows = mysqli_fetch_all($res) is executed, first $rows gets created. Then, mysqlnd makes $rows reference the mysqlnd internal result buffers. MySQL results are not copied initially. Result set data is kept only once in memory.

PHP script mysqli extension/API mysqlnd library MySQL
*.php *.c
  store_result()  
  Buffer with zvals (

Truncated by Planet PHP, read more at the original (another 16365 bytes)

Link
Bruno ŠkvorcHow to Speed Up Your App’s API Consumption (10.4.2014, 16:00 UTC)

In the process of creating a PHP application you may come to a point when keeping it isolated from remote resources or services may become a barrier in its development. To move along with the project you may employ different API services to fetch remote data, connect with user accounts on other websites or transform resources shared by your application.

The ProgrammableWeb website states that there are currently more than ten thousand APIs available all over the web so you’d probably find a lot of services that can be used to extend your PHP app’s functionality. But using APIs in an incorrect way can quickly lead to performance issues and lengthen the execution time of your script. If you’re looking for a way to avoid it, consider implementing some of the solutions described in the article.

Continue reading %How to Speed Up Your App’s API Consumption%

Link
PHP ClassesOpenSSL Serious Security Bug: Does it Affect Your PHP sites? (10.4.2014, 09:39 UTC)
By Manuel Lemos
Just a few days ago it was publicly announced a serious security bug called Heartbleed that affects secure sites based on the OpenSSL library.

Read this article to learn more about this security problem, how to test if your Web server or SSH server is vulnerable, how it may affect your PHP sites, what you should do to fix the problem.
Link
Bruno ŠkvorcIntroduction to JadePHP (9.4.2014, 17:32 UTC)

There are dozens of templating engines out there, with options such as Smarty, Twig (used in the upcoming version of Drupal) and Blade (the default for Laravel) among the best known - as well as vanilla PHP, of course. Stepping away from PHP specifically, eRuby / ERB and Haml for Ruby / Ruby on Rails, and Javascript has scores of popular choices including Mustache, Handlebars, Hogan and EJS. Some have subtly different syntax, some more markedly so.

One which differs quite significantly from most is Jade, an engine usually associated with Javascript applications - it’s supported out-of-the-box by Express for Node.js, for example. It’s Jade I’m going to look at in this article; or more specifically the PHP port JadePHP.

Haml and Jade

It would be remiss to talk about Jade without mentioning Haml, from which Jade takes its inspiration - and indeed there are several libraries for using Haml with PHP. Jade shares its overall philosophy, which is to make templating “beautiful” and use what the authors describe as templating “haiku”. Whatever that actually means, there’s no denying Haml and Jade do share some characteristics which make them fundamentally different to most templating languages.

What’s the Difference?

Most templating engines involve writing the target markup and “injecting” it with placeholders and / or basic logic - a superset, in a sense. Jade still has placeholders and logic, but also provides a shorthand for writing XML-like elements. Generally that means HTML, although you can also use it for things like RSS as well as XML itself.

In fact if you wanted to, you could just use Jade as a shorthand for HTML without taking advantage of its more “traditional” templating features.

How to use the Repository

Rather frustratingly, the code is not currently available via Composer - although it should be a simple enough task to package it up, if anyone has an hour or two. You can get it to to work, however, by cloning the repository and include‘ing or require‘ing the included autoload.php.dist (the Github repository includes Symfony’s UniversalClassLoader).

Here’s an example, adapted from the one in the project’s README, which assumes that the repository has been downloaded into a directory called jade:

require('./jade/autoload.php.dist');

use Everzet\Jade\Dumper\PHPDumper,
        Everzet\Jade\Visitor\AutotagsVisitor,
        Everzet\Jade\Filter\JavaScriptFilter,
        Everzet\Jade\Filter\CDATAFilter,
        Everzet\Jade\Filter\PHPFilter,
        Everzet\Jade\Filter\CSSFilter,
        Everzet\Jade\Parser,
        Everzet\Jade\Lexer\Lexer,
        Everzet\Jade\Jade;

$dumper = new PHPDumper();
$dumper->registerVisitor('tag', new AutotagsVisitor());
$dumper->registerFilter('javascript', new JavaScriptFilter());
$dumper->registerFilter('cdata', new CDATAFilter());
$dumper->registerFilter('php', new PHPFilter());
$dumper->registerFilter('style', new CSSFilter());

// Initialize parser & Jade
$parser = new Parser(new Lexer());
$jade   = new Jade($parser, $dumper);

$template = __DIR__ . '/template.jade';

// Parse a template (both string & file containers)
echo $jade->render($template);

This will compile the file template.jade and echo its contents.

Where you actually use this depends on your workflow, whether you’re using a framework, and so on. You could, perhaps, use a service such as Watchman, Guard or Resource Watcher to watch the filesystem for changes to your Jade templates, and compile them at the appropriate time during the development process.

Continue reading %Introduction to JadePHP%

Link
Derick RethansCursors and the Aggregation Framework (9.4.2014, 08:29 UTC)

Cursors and the Aggregation Framework

With MongoDB 2.6 released, the PHP driver for MongoDB has also seen many updates to support the features in the new MongoDB release. In this series of articles, I will illustrate some of those.

In this article, I will introduce command cursors and demonstrate how they can be applied to aggregations. I previously wrote about the Aggregation Framework last year, but since then it has received a lot of updates and improvements. One of those improvements relates to how the Aggregation Framework (A/F) returns results. Before MongoDB 2.6, the A/F could only return one document, with all the results stored under the results key:

<?php $m = new MongoClient;
$c = $m-?>demo->cities;

$pipeline = [
     [ '$group' => [
          '_id' => '$country_code',
          'timezones' => [ '$addToSet' => '$timezone' ]
     ] ],
     [ '$sort' => [ '_id' => 1 ] ],
];

$r = $c->aggregate( $pipeline );
var_dump( $r['result'] );
?>

This code would output something like:

array(242) {
  [0] =>
  array(2) {
     '_id' => string(2) "AD"
     'timezones' => array(1) { [0] => string(14) "Europe/Andorra" }
  }
  [1] =>
  array(2) {
     '_id' => string(2) "AE"
     'timezones' => array(1) { [0] => string(10) "Asia/Dubai" }
  }
  [2] =>
  array(2) {
     '_id' => string(2) "AF"
     'timezones' => array(1) { [0] => string(10) "Asia/Kabul" }
  }
  …

MongoCollection::aggregate() is implemented under the hood as a database command. The method in the PHP driver merely wraps this, but you can also call A/F through the MongoDB::command() method:

<?php $m = new MongoClient;
$d = $m-?>demo;

$pipeline = [
     [ '$group' => [
          '_id' => '$country_code',
          'timezones' => [ '$addToSet' => '$timezone' ]
     ] ],
     [ '$sort' => [ '_id' => 1 ] ],
];

$r = $d->command( [
     'aggregate' => 'cities',
     'pipeline' => $pipeline,
] );
var_dump( $r['result'] );
?>

Because a database command only returns one document, the result is limited to a maximum of 16MB. This is not a problem for my example, but it can can certainly be a limiting factor for other A/F queries.

MongoDB 2.6 adds support for returning a cursor for an aggregation command. With the raw command interface, you simply add the extra cursor element:

$r = $d->command( [
     'aggregate' => 'cities',
     'pipeline' => $pipeline,
     'cursor' => [ 'batchSize' => 1 ],
] );
var_dump( $r );

Instead of a document with all results inline, you get a cursor definition back:

array(2) {
  'cursor' =>
  array(3) {
     'id' => class MongoInt64#5 (1) {
          public $value => string(12) "392201189815"
     }
     'ns' => string(11) "demo.cities"
     'firstBatch' => array(1) {
       [0] =>
       array(2) {
          '_id' => string(2) "AD"
          'timezones' => array(1) { [0] => string(14) "Europe/Andorra" }
       }
     }
  }
  'ok' => double(1)
}

The cursor definition contains the cursor ID (in id), the namespace (ns), and whether the command succeeded (in ok). The definition also a portion of the results. The number of items in firstBatch is configured by the value given to batchSize in the command.

To create a cursor that you can iterate over in PHP, you need to convert this cursor definition to a MongoCommandCursor object. You can do that with the MongoCommandCursor::createFromDocument() factory method. This factory method takes three arguments: the MongoClient object ($m in my example), the connection hash, and the cursor definition that was returned. The hash is required so that we can fetch new results from the same connection that executed the original command.

To obtain the connection hash, we need to include a by-ref variable as the third argument to MongoCollection::command():

<?php $m = new MongoClient;
$d = $m-?>demo;

$pipeline = [
     [ '$group' => [
          '_id' => '$country_code',
          'timezones' => [ '$addToSet' => '$timezone' ]
     ] ],
     [ '$sort' => [ '_id' => 1 ] ],
];

$r = $d->command(
     [
          'aggregate' 

Truncated by Planet PHP, read more at the original (another 2607 bytes)

Link
Liip My HHVM talk slides from yesterday's Webtuesday (9.4.2014, 06:39 UTC)

Yesterday I gave a talk about HHVM and Hack at the monthly webtuesday here in Zurich. The slides can be found here

It was a great evening, thanks to Netcetera for hosting it, thanks to Joel and the webtuesday crew for organizing it and also thanks to Johannes for chiming in when we talked about PHP internals.

If you have any question abut HHVM or related things, just ask in the comments, contact me on twitter or meet me personally, if you're nearby (aka in Switzerland).

Link
Brandon SavageThree Great New Speakers! (9.4.2014, 00:24 UTC)

This summer’s Crafting Code Tour is coming together, and we have confirmed three speakers to come along to various locations!

We are proud to announce that Adam Culp will be joining the tour for stops in Tampa Bay, Atlanta and Nashville. Adam will be speaking on Refactoring 101, offering up his wisdom on how to refactor legacy applications.

Paul M. Jones will be joining the tour for Dallas, Austin, Oklahoma and Kansas City, speaking about code modernization, a great talk that he’s given at numerous conferences in the past year.

And Larry Garfield will join the tour for Minneapolis, Milwaukee and Cincinnati. He’ll be doing a reprise of his SunshinePHP keynote, so if you missed it, don’t miss it on these stops!

Plus, we are pleased to announce that Mandrill will be providing pizza for all the tour stops.

Link
Symfony CMFFinally! 1.1.0-RC1 (8.4.2014, 04:00 UTC)

Once we released 1.0, we did not sit idle. Instead we immediately began work on improving the CMF. In fact there was such a continuous stream of good ideas and things that felt like a must have, that we let ourselves slip quite far from our defined release process. Or lets say we had every intention of aliging ourselves with the Symfony2 core release cycle by releasing within 1-2 months of core which would have meant a 1.1 release in January. Instead it tooks us 6 months to get to RC1, which on the upside is inline with what we intended for future releases.

As we feared, it turns out that we had to do a few minor BC breaks which we tried to document as much as possible in the CHANGELOG.md files in the respective packages. We also have updated the documentation already with lots of details on the new features we added. Unfortunately due to limitation in the current symfony.com website, we cannot provide a rendered version just yet. So for now please review the dev branch. Hopefully we will eventually also be able to provide multiple versions of the CMF documentation just like it is possible for the core documentation.

Let me briefly highlight some of the key improvements that are coming with 1.1:

  • A lot of effort was spend in making the CMF more resilent towards edge cases and given better more helpful error messages.
  • Support for newer version of Jackalope and PHPCR ODM, that bring lots of bug fixes, speed improvements, and better debugging and profiling (including Symfony2 debug toolbar support).
  • The routing system was made even more flexible, meaning there is even less code to write when implementing a new storage backend. As a result much less code is now needed in SimpleCmsBundle for routing.
  • The menu system has seen several tweaks, most noteably it is now possible to configure a fetch depth, which can give drastic performance improvements for deeply nested menu structures.
  • A new Bundle was added for handling SEO meta data.
  • A totally reworked Standard Edition that is based on the core Standard Edition to make it easier to understand what is necessary to add to a Symfony2 application to integrate the CMF.

So what is still left until we can go stable? For now we want to give at least 2 weeks from now to allow the community to do some testing and to focus on bringing all the documentation up to date. So expect a stable release at the earliest towards the end of April at the latest sometime in May. Other than this the main areas where we might still see some changes are:

To get an idea of what is needed to update to 1.1, have a look at the PR to update this website to the new version. Also note that the CMF sandbox has also already been upgraded to 1.1.

We are also very happy with the growing eco-system around PHPCR. Most noteably there is now a PHPCR shell to interact with a PHPCR repository as well as finally a PHPCR GUI that supports reading and writing via a web interface!

Link
Bruno ŠkvorcGetting Started with PHP Extension Development via Zephir (7.4.2014, 17:00 UTC)

Thien Tran Duy explains how we can get started with PHP Extension development through Zephir, the new language and direct competition to Hack and HHVM from the Phalcon team

Continue reading %Getting Started with PHP Extension Development via Zephir%

Link
Paul M. JonesComposer-Assisted Two-Stage Configuration in Aura (7.4.2014, 16:06 UTC)

After a long period of consideration, research, and experiment, we have found a non-static solution for programmatic configuration through a DI container. It is part of a two-stage configuration process, implemented through a ContainerBuilder.

The two stages are “define” and “modify”:

  • In the “define” stage, the Config object defines constructor params, setter method values, and services. This is the equivalent of the previous single-stage Solar and Aura v1 configuration system.

  • The ContainerBuilder then locks the Container so that its definitions cannot be changed, and then begins the “modify” stage. In this second stage, we retrieve service objects from the Container and modify them programmatically.

So now, instead of each Aura package carrying a pair of define and modify includes in a subdirectory named for the config mode, we have a single class file for each config mode. The class file is in a subnamespace _Config under the package namespace. Here are two examples, one from the Aura.Web_Kernel package, and one from the Aura.Web_Project package.

Because they are classes, you can call other methods as needed, subclass or inherit, use traits, create instance properties and local variables for configuration logic, and so on. You could even call include in the methods if you wanted, keeping the class as a scaffold for much larger configuration files. This gives great flexibility to the confguration system.

To make sure the Aura project installation can find all the package config files, we use the {"extra": {"aura": { ... } } } elements of the composer.json in each package to provide a mapping from the config mode to the config class.

Read the whole article at Composer-Assisted Two-Stage Configuration.

Link
LinksRSS 0.92   RDF 1.
Atom Feed   100% Popoon
PHP5 powered   PEAR
ButtonsPlanet PHP   Planet PHP
Planet PHP