Archive for the ‘PHP QA’ Category

SQLI_CodeSniffer goes PHP_CodeSniffer

Sunday, January 3rd, 2010

Part of my daily work consist in assisting companies using PHP to improve their quality standards and methodologies.

Large organizations and big IT services companies I’ve worked for already have their own coding style rules with multiple severity levels specified in official documents.

These documents are supposed to establish norms for all internal and external developments. They are subject to annual reviews in which QA people meet developers to decide evolutions and parameters and severities changes: “Should we enforce this rule?” “Should we allow for weaker conditions on that?”

When I install a continuous integration system for this kind of client, the main effort goes in implementing the biggest possible part of these standards. Of course my French clients would really appreciate to find their own-defined codes and (French) messages in reports. Even more they would like to find their severity levels respected.

The tool that allows me to do this is PHP_CodeSniffer, a well established PEAR package created and maintained by Greg Sherwood. Costumization is done by creating a new PHP_CodeSniffer “Standard” and writing down some PHP classes called “Sniffers” that actually detect coding rules violations.

While I find great, and underused, the possibility to programmatically create your own sniffers, the task is not straightforward. Fortunately PHP_CodeSniffer comes with a large set of implemented sniffers, so what you normally do is to inherit from existing ones. If parameters are well separated in the parent Sniffer properties and you only need to change them, you can simply override their values.

Problems come when you want to change severities and messages, as they are someway hardcoded in the Sniffer. You’re left with the choice of duplicating the existing Sniffer’s code or to leave the original message and severity.

Note that as messages are dynamical (they contains variables, for example the name of the class under inspection) it’s impossible (or really painful and error-prone) to recover the error “a posteriori” from its message.

To overcome all these difficulties I had the idea of writing a wrapper around PHP_CodeSniffer to overload the way errors and warnings are thrown and to allow a more flexible report generation. As I’m not gifted in finding names, I called it SQLI_CodeSniffer :-)

Its main ideas can be synthesized as :

  1. Letting sniffers add a violation code to every error/warning they throw.
  2. Delay the  severities and messages attribution at report time.
  3. Let them be taken from a configuration file matching on the newly added identifiers.
  4. Make it easy to add a new, user-defined type of report.

Two main advantages this new approach brings are that you can now:

  1. add more severities levels and filter reports up to a threshold,
  2. collect statistics about your violation types.

When I firstly saw the Sonar interface (look at my previous post for more) I understood that what I had done with SQLI_CodeSniffer could easily be adapted and furnish all that was needed for that kind of visualisation: just add the code violation to the report, add a category to that kind of violation in the configuration file and give that file to Sonar.

The problem with SQLI_CodeSniffer is that it is deeply tied with the original package.  It’s really like a sort of big hack. Every modification done to PHP_CodeSniffer risks to make it unworkable. So I hesitated to publish it.  I did indeed, but privately, here.

But this story has an happy ending, or should I say an happy new beginning.

Greg Sherwood took a look at SQLI_CodeSniffer, liked it, and decided these functionalities were worthy the main package.

So we decided to join efforts and port this and more back into PHP_CodeSniffer. All the work will be avalaible in the 1.3.0 release, we will be really careful of backward compatibility.

Greg explains the new features with nice examples in a post on the new PHP_CodeSniffer blog.

SQLI_CodeSniffer is dead, long live PHP_CodeSniffer!

Sonar for PHP is on its way

Thursday, December 31st, 2009

It’s no secret that we at SQLI want to give the PHP world an Open Source full-featured continuous integration system.

As an essential step to this goal in the short term, we are working on enabling the Sonar QA reporting system for PHP projects.

Sebastian Bergmann asked some time ago for this possibility, but he discovered one missing link to complete the chain: some Java plugins to read and integrate PHP reports into Sonar.

Fortunately at SQLI Paris we have a strong Java department and Jérôme Tama volunteered to help in this task, supervised by Frédéric Leroy.

If you look a little into the Sonar WebUI you’ll see that a lot of information is brought in Java projects by the checkstyle plugin.

Unfortunately the possibilities of our analog PHP_CodeSniffer are actually smaller, giving a poorer Sonar report. This was the other missing link, less obvious to see.

The good news is that I was working since a while on “SQLI_CodeSniffer”, a wrapper to PHP_CodeSniffer that adds most of the features needed for a full integration.

So we’ve been able to integrate reports from SQLI_CodeSniffer, PHPUnit with XDebug code coverage, PHP_Depend, and the PHPUnit pmd report (partially added, we plan to switch to the brand new PHPMD).

So today I’ve a nice prototype working on my computer and I want to share some screenshots with you.

On the Dashboard screen you have a rich project overview with a remarkable radar of rules compliance by nature.

On the Components screen you have a view by component with an impressing treemap visualization. Let’s play with it !

You can change the measure represented by the size of rectangles. Let’s choose complexity.

And the measure that gives their color. Let’s choose coverage.

Ok, now I have a synthetic view of the project testing effort areas and priorities to cover that complexity.

Every report in Sonar can be specialized by component:

Now I see an overview for my project controllers.

In the component view I now have details for each class instead.

On the Violations drilldown screen you have detailed statistics about severities of your violations, most violated rules, violations by component and by class.

You can also see violations filtered by categories. You can easely concentrate your improving efforts on certains areas first.

Selecting a file gives you the code details with violations just pointing to the incriminated line: clear!

You can also filter the shown violations.

Switching to the coverage tab gives you the familiar covering visual report, with the numbers on the left indicating how many times your tests passed on the line.

The duplications tabs gives you details about your file code duplications. You can see the duplicated code by expanding the line.

In the Time machine screen you’ll be able to select all of your measures and see their evolution through time. In the greed you can choose for which dates to show figures by selecting them directly or filtering them by events.

In the Hotspots screen you have a nice synthetic view of project priorities: most violated rules, most violating classes, most complex classes and methods, …

I’ve not played enough with the administration part, but just looking at the menu you can imagine the richness: users/groups, global and project roles, defining manual metrics… Let’s just jump into the PHP quality profile.

I choosed the SQLI_CodeSniffer rules (PHP_CHECKSTYLE). I’m able to edit their presence and severity through the Sonar interface. This just let me think about all the times I had applied some good PHP_CodeSniffer standard on projects I was auditing just to discover tenths of thousands errors… What to suggest to developers? How to help them concentrate their effort?

My first impression as a user: Sonar informational richness and reporting quality is so superior to what we’re accustomed to see that this can really represents a jump in usability and in QA everyday practices.

If you like this as much as I do, you’ld probably go and download SQLI_CodeSniffer and search for the Sonar plugins… but please, be patient for a little while.

Development is still in alpha as important changes are coming, the most important being the merge of SQLI_CodeSniffer into PHP_CodeSniffer, and there is no sense in publishing all the work right now.

I’ll talk about all of this in an upcoming post: stay tuned!