WordPress theme/plugin directory brainstorming – quality assurance

A post over at CMS Critic received some unwarranted flack due to how it was written. However if you can look beyond any fanboyish reactions, some issues the author, Kaya Ismail, brings forth is worth taking some time to think and write about.

What I’ll focus on in this post is the vast plugin directory. Kaya writes a bit about the issues end-users have regarding selecting and installing plugins that don’t actually work. There are multiple reasons as to why a plugin does not work on a site; such as: PHP code errors, permissions issues and plugins conflicts. I’ll write a bit on various ways that WordPress itself and wordpress.org can do to prevent users from installing non-functioning plugins. I’m not taking things like, user experience, performance, scalability, DDOS scenarios, and more into account. There might be potential privacy issues with some solutions as well. Also this is just about reporting non-functioning plugins and I won’t discuss improvements to search and listing of plugins. This post is just a brainstorming session.

Manual feedback – Current approach

The current feedback system is manual. Plugin and theme users needs to visit the plugin page in the directory and mark if the plugin works or not. The biggest issue with this is that very few people actually takes the time to do this even when a plugin is broken. The second biggest issue is that as an end-user you have no idea if the reports are about actual issues with the plugin or that it didn’t do all the things the reporter wanted it to do. There is not much one can do about the second issue but there are possible solutions to the first. One way to get information is to ask the user when they deactivate the plugin. “Did it work for you? Yes, No”. If user chooses “No” show a field for additional information. To limit when it shows up WordPress could disable it when using WP_DEBUG. Would be annoying to have it show up if you are working with activation/deactivation issues as a developer.

Of course one issue with asking users upon deactivation is that it can increase the amount of reports that concern the user expectation and not if the plugin actually works or not. Adding radiobuttons; had errors, didn’t do what I wanted, my site looked weird, to the pop-up form could be one way to counter the false reports. If alot of people report issues but the plugin is not broken that should be cause to review the plugin description.

Solutions for WordPress.org

Automatic checks on WP.org

Automatic code validation on .org could detect minor issues but the amount of false positives would probably outweigh the usefulness on this approach.

Other ways to detect issues would be to activate each plugin on a test site before making it available to users. Having plugins activated and then run updates on them would detect any problems before the issues reach the plugin users. After activation visits to front end and various backend pages could be made and if any PHP errors are detected the plugin updates would not be going out.

Manual checks on WP.org

This is probably the most cumbersome solution. What this would entail is that a group of testers manually verify if plugins are working as expected. They go to a sandbox install on wp.org and activate and do some rudimentary testing of functionality of a plugin. Then they mark the plugin as working or not working. Plugin downloads/updates are only available to all users after it has gone through this process. You should be able to opt-in to use untested plugins/themes if you are adventurous otherwise you cannot activate the plugin on your own site. A curated list of plugins of sorts. It could even be service that end-users pay for. This would work best if WordPress by default supported multiple plugin and theme directories.

Automatic checks on end-users sites

Most efficient reporting systems will rely on actual user usage of plugins and themes. There are a number of scenarios where WordPress, in theory, could detect issues and report them.

Activation issues

This one is easy enough. WordPress already activates plugins in a
sandboxy way and detects errors upon activation. One way to improve the plugin/theme directory is to send this information to wp.org along with which plugins and theme that is used. When the WP service has registered some percentage of activation issues compared to installs it marks the plugin as broken. This information can then be emailed to the developer so they can act or not act. While waiting for a new update the directory will show the plugin as broken and inform the user of the risk. Of course if all activations generates errors then the plugin will not be available to be installed at all. The verify upon activation approach also gives the added benefit that plugin authors have to think twice about doing time-consuming tasks during activation or perform actions that require specific file permissions. Those two cases often cause errors for users.

Runtime issues

This comes into effect after some event, usually update/activation of a plugin or theme. Some plugins breaks the site since they generate an error. With regards to runtime errors there are 3 types that are most common. From the PHP documentation:

  • E_ERROR: Fatal run-time errors. These indicate errors that can not be recovered from, such as a memory allocation problem. Execution of the script is halted.
  • E_WARNING: Run-time warnings (non-fatal errors). Execution of the script is not halted.
  • E_NOTICE: Run-time notices. Indicate that the script encountered something that could indicate an error, but could also happen in the normal course of running a script.

A plugin or theme should not generate any of those types of errors. If it does then it should not be used since the quality of the plugin/theme cannot be trusted. And by trust I include security issues as well. If your plugin generates warnings and notices it is not unlikely that it also contain potential security issues due to not following best practices. Of course sometimes notices occur due to WordPress not handling certain scenarios correctly but that’s something for another time. However the only one of the errors that actually can stop a site from functioning all together is E_ERROR. If a plugin generates such errors during runtime it most definitely should not be installed by anyone.

The question then is how can we make sure that these types of issues don’t make it to the end-user? The first things that comes to mind for me is to extend the current error handler. WordPress already comes with an error handler implementation but it is only used during activation. Having this run during normal scenarios would make it possible to catch errors, detect which plugin that causes the issue and send a report to WordPress. The information that could be sent is the error itself, WordPress version, plugins and theme. Also the PHP and MySQL version might be good to collect. If WP.org has this info it can then mark the plugin as breaking if enough sites reports issues. It’s also possible to inform the developer of the issues before the first angry people visits the support forum. Using a custom error handler causes issues with the default logging since its replaced and I’m not sure how to best handle the conflicts with monitoring services that arise from that. One thing is probably certain, those who use monitoring services will probable disable this feature all together anyway.

In conclusion

Well, there are quite a few different approaches to the issue of non-working plugins and themes in the wp.org directory. Some are doable others not so much. I have just tried to come up with a bunch of ideas. Do you have more ideas on how to make sure that the plugins and themes users download actually work properly? Are there any of my ideas that you wish to expand upon go ahead and do so in the comments.


Customizer – the good, the bad and the ugly


The Customizer in WordPress has gotten quite a lot of critique lately mostly concerning a misunderstanding about the proposed removal of the current Menu interface under Appearance in favor of just having Menu management in the Customizer interface. This proposal was rejected by the WordPress Core team which made many of us very pleased. The new Menu interface is still going into WordPress in a very fast track manner. Almost as the massive negative feedback has given them more energy to push it through. We will see if it actually gets into 4.3 or not my bet is on yes. In this post I have tried to find out everything I can about the origins of the Customizer.

The new Menu management is developed as a plugins first approach which was introduced back in mid 2013. This new way of adding features to WordPress Core is really good and I was pleased that the WordPress Core team went with that development procedure.  Ryan McCue (@rmcure) and the WP-API project he initiated and have spearheaded despite not being part of core is an excellent demonstration of how Core should always be developed for any major changes. My cynical side would have thought that REST API support in WordPress Core would have come from a conversion of the current XML-RPC implementation. Ryan should be praised for not taking that approach and for taking the matter as seriously as he has and his ability to get Core leadership on board and the rest of the team (Rachel Baker, Joe Hoyle, Daniel Bachhuber).

My approach regarding these posts concerning the Customizer is to first try to figure out and explain where it all stems from, why this approach was bad and how it has impacted and will continue to impact the development of WordPress as a project. I will also try to show the detrimental adhoc nature of WordPress Core development.

Customizer – The origin story

The Gandalf incarnation

So what does the Customizer really have to do with my appraisal of Ryans et als work with the WP-API? Well the Customizer is from my perspective the exact opposite of how one would go about handling such big pervasive changes. There was a plugin approach used for the Customizer development, one could find it under the trac ticket #19910, it remained a plugin for 25 days. More on that later. I cannot find any public discussions on design or approaches on how to best implement a standardized interface for theme manipulation and related settings. There are mentions of themes and customizing in an old post from a in person Core Meetup back in late 2012. Other concrete stuff is scope discussions for 3.4

High-level, the features would likely include: a theme-setup wizard that would incorporate an option for configuring all the appearance-related stuff before activating a new theme (speaking of, Twenty Twelve is targeted for 3.4), and then specific improvements around menus, widgets, backgrounds, headers, easier static front page process, multisite appearance management, etc.

Tracing forward we have the “This Is Not a Feature List” post on January 11, 2012 where we find the following list:

Framework for configure and activate (theme + associated custom header, background, menus, widgets) – Koop, Ocean

  • live preview of theme changes
  • activate without configure
  • drag and drop sidebars/widget areas from old theme to new theme
  • configure a new theme, with preview, and then push that theme live
  • easier static front page process

Next update regarding this whole ordeal comes on February 3, 2012 with Team Gandalf Update. No updates at all between the first mention on January 11th and this update, even though the post claims they are halfway through their first iteration. There was no post to mark the start of it in the first place except the “This is not a feature list” post. This is the origin of the whole Customizer feature. No screenshots, no updates, no various design suggestions, no public debate regarding anything except for some code implementation issues on the trac tickets. And it was all developed under the very explanatory name of “Team Gandalf”. Why they didnt name themselves The Theme Editor Editors or something fun but explanatory we can only guess (or ask them but I’ve got much more to write about).

First iteration ended on February 10, still no elaboration on the design choices. The trac tickets is still lack luster without any screenshots or design ideas. There are discussions on code implementation. In 15 days the Customizer plugin gets merged into core with the message “Introduce new theme customizer to replace theme preview. Rough first pass.”. A whooping 25 days from first introduction of the plugin and the ideas getting merged into core. 

Yesterday marked the halfway point of our second cycle. We’re moving along at a steady clip. The main goal for the coming week is to tie up any loose ends and begin integrating the plugin into core. Until then, follow our progress at #19910 and in the plugin repo.

The lasts posts from the Gandalf teme is Update 3 and Update 4.

In the original ticket Appearance Improvements: An Overview #19909 we can find a pretty lofty goal.

The main target for the 3.4 appearance improvements will create a UI that allows users to simultaneously customize and preview their theme. Our initial target will be the theme switching process (in an a la carte fashion), which will then hopefully expand to more uses in the WordPress admin.

So essentially the whole Customizer interface stems from an idea of listing themes and choosing one to preview even though the original goals were much greater. Theme selection using the  Customizer works very good for the most part.

The earliest display of what the Customizer will look like that I can find is a tweet by Mark Williams. Next up is a “How to leverage the Theme Customizer in your own themes” post by Otto on May 3, 2012. I cannot find a single screenshot on the make blog or in the trac tickets. If they exist I would gladly add them to the post.

Present day


Quick example of Theme Previewer

The last few weeks we have had lots of discussions in the community about changes to the Theme repository with the requirement of using the theme customizer. If you check the comments on the various posts regarding the Customizer in the past on what was wpdevel.wordpress.com you clearly see that people either didn’t know about it or didn’t cared enough to comment. I think just adding a mockup, like the one I made in like 5 minutes, would have helped greatly in getting people on board the Customizer train. Just such a simple thing would have increased the feedback greatly I think. And adding some mockups for how it would be used for modifying theme design etc would most likely have resulting in a “NO” response from the parts of the community that have any understanding of front end editing such as all the Drag and drop theme developers and users.

The lack of community feedback during the development of the Theme Previewer is most likely the main culprit for why the Customizer was rarely used by themes in the past and also why some are reluctant to use it now. People are coming aboard the train but very few actually seem to glad to be on the train. I’ll be covering various themes, plugins that make use of the Customizer and comparing them to other competing services.

It is 3 years since the Customizer made it into core without community feedback, I cannot help but wonder if they had taken a more integrated and open approach similar to how Ryan et al has done and continues to do with the WP API we might have had a totally different experience.

The good thing about the latest developments of the Customizer and other parts of the core development is that its starting to get more public. The posts relating changes to Core has improved in quality and also most things relating to anything visual actually contains screenshots or GIF animations of how things will look and work. Personally I would really like to see more posts about the steps leading up to the visual implementation of features not just the end result or what you might call the Release Candidate. The bart of all is that the Core ignored the vocal opposition, failed to address major questions that came to light given these changes. The WordPress Core team have not seen such a backlash since the introduction of the infamous capital_P_dangit() function back in 2010. Justin Tadlock wrote the following regarding that change:

Listen to your community

I’ve only seen a handful of people that agree this function should be in WordPress. It has been met with harsh opposition. The people arguing to remove this function are people we need in the community. They’re plugin developers, theme developers, contributors to core code, and evangelists for the WordPress platform. These are the people that continue making WordPress better.

Don’t alienate them.

Are these people only a vocal minority of WordPress users? Certainly. However, these people speak for a larger amount of users. For users without the knowledge of mailing lists. For users without the understanding of how Trac works. They are the people that interact with the majority every day. They are the voice of the majority.

Don’t let their voices go unheard.

Justin Tadlock,


We can now deduce that the Customizer looks the way it does because it all started with the single feature of previewing and selecting a theme. The constraints regarding that single feature is what have led us to a Customizer that looks like the current one. Why the feature of selecting a theme should dictate how the Customizer looks and behaves for all other theme and design related controls and behaviors is something that has not been properly argued by the Core team. There is not a single mockup and so forth showing that this is the best way going forward. They wanted something and added it, with a total disregard for the future or the endusers and completly ignoring the community backlash.


WordPress plugins/themes and backwards compatibility

Pippin Williamson of pippinsplugins fame did a presentation on backwards compatiblity and WordPress plugins at LoopConf. In the presentation he showed a couple of examples of where developers have a tendency to break backwards compatibility. In this article I will give my thoughts on what he said and also give some ideas on how you can prevent breaking things in the first place.

Hide how things are stored

In the presentation Pippin gives an example of get_post_meta and changing the key used for storage or how its stored all together like a taxonomy or separate table.

$meta_values = get_post_meta( $post_id, $key, $single );

This is not information that should be available to the developer. The developer should not know how things are stored or even have to care. It is not something they in most cases need to know. The value should be accessed using a wrapper function or be a property of a class. Some developers makes the case that developers should not know about the classes themselves and argue that only basic structures should be used to communicate data, so stdClass, arrays and so on. Instead of returning a class with properties you return an array representation of the information. A minimal class without functionality and with public properties can also be used. This has the advantage over arrays and stdClass of providing autocompletion when using code editors. They are also easier to document.

Don’t encourage direct inclusion of files

If its possible, try to use autoloading of your classes. You can also add a wrapper function that includes a file containing functions since functions cannot be autoloaded. The files should preferably be included by default on load. Doing require_once on all the the plugin files is not a recommended approach and its not something developers should do on your plugin files. If you have to use require_once to include files you have a bad structure in your plugin or theme. Having wrapper function to include file and also discourage use of require_once? Yes well I like paradoxes. Run tortoise run.

Don’t hard code any HTML code in your plugin

Pippin says to be carefull with HTML structure in plugins since you can break extensions later on if you change the tags and so forth. However there are ways to minimize this risk. One way is to provide filter for wrapping tags etc if you are forexample showing a list.

$wrapper = apply_filters('list-wrapper',['parent'=>'ul', 'tag'=>'li'], $context)
echo "<$outline>";
foreach ($items as $item) {
  echo "<$tag>", 'asdfasdf', "</$tag>";
do_action('list', $wrapper, $context);
echo "</$outline>";

Tag inclusion can be used in conjunction with actions as well.

<?php foreach ($items as $item) { ?>
<?php }?>
<?php do_action('list-item', $context,
  '<li class="'.implode(' ',apply_filters('li-classes',array(),$context)).' %s">',
//do_action('list-item', $context, '<li class="%s">%s</li>');

Of course the above methods does not work if you go from one column of data to multiple columns of data. Such as going from a list with singly value to a table with two column layout with more data displayed. One way to handle variable columns is to use a placeholder system. This works as long as the objects to display all adhere to the same contract. I personally loath plugins that prints hardcoded HTML. Its my data I should decide how its displayed.

Don’t hardcode HTML etc in JavaScript files

JavaScript is usually a must in all modern sites. However how you structure your JavaScript can have huge effects on both extendability and maintaining compatibility. This point is in the same vein as the previous one. Usually WordPress themes and plugins makes use of jQuery however they use it badly. Or rather they make use of JavaScript badly. In order to make it easier for you and your eventual endusers try to separate out all DOM manipulation from your events. So if you have for example a button “Add to list” which stores an item on a list on the server do not include any DOM manipulation in that event. Separate the responsibilities.

//<ul class='somelist'></ul>
$('.add').on('click', func(e){
  var data = retrieve_data('list-entry-form');
  $.post(url, data).success(func() {$('.somelist').append('<li>asdasd</li>'}).done(func(){button.enable();});

If a theme, plugin or you have changed the tag for that list to a div now you have breaking HTML. Instead try to make use of events so you can better separate the various parts of your code to separate files. It also enables any themes or plugins to remove your default DOM manipulating code and replace it with their own. Of course this is a two-edge sword. To separate responsibilities you can make use of jQuery trigger() and on() functions.

//my-actions.js handling posting data and triggering events
$('.add').on('click', function(e){
  var data = retrieve_data('list-entry-form');
  var button = this;
  $(document).trigger('list-item-adding', [button,data]);
  $.post(url, data).success(function() {
    $(document).trigger('list-item-added', [button,data]);
  }).done(function() {$(document).trigger('list-item-done', [button,data]);});
//my-subscribers.js subscribing to events and manipulating DOM
$(document).on('list-item-adding', func(button, data) { $(button).disable()});
$(document).on('list-item-added', func(button, data) { $('somelist').append('<li>somestuff</li>'});
$(document).on('list-item-done', func(button, data) { $(button).enable()});

If you want you can even make the class .add customizable using a filter and making use of the wp_localize_script function to print the class, ID to use for the on Click event. Separating the responsibilities is also could if you want to make use of translatable JavaScript strings.

Write unit and functional tests

The fastest way to find out if things have changed and might affect endusers and other developers is breaking tests. If you have written tests for your code and they fail when you make changes you really need to think about the changes you are making. This is good both for your JavaScript code and for your PHP code. Tests can be a lifesaver and decrease the amount of support requests you get.

Write acceptance tests

To make sure things are working when it comes to JavaScript, classnames, structure write acceptance tests. This is usefull both for the website and for the WordPress admin. You need to make sure your settings page behaves correctly as well. There are numerous frameworks for this type of testing. You can use PHPUnit with Selenium, Codeception or why not use a JavaScript framework like CasperJS. When it comes to the frontend tests they can be made available to other developers as well so they can test their plugins of your plugin or theme.

Let the developers know

Write good documentation of your code and how its meant to be used. If certain class properties are meant to be read only use @property-read. If a class or function is meant only to be used by the plugin itself mark it as internal. If developers disregards the instructions that is not your concern. Provide the developers with a best practice extension so they can see how things are meant to work.

Incremental changes

If you forsee breaking changes you can do as Pippins says in the talk and introduce the changes version by version. It all depends on what kind of changes you are making. This ofcourse relies on the endusers actually updating the plugin to each version in the first place.

When backwards compatiblity can’t be kept

Depending on how big the changes are you might be better of forking your own plugin or theme if the changes are really big. If you do not want to do that and still want to make bigger changes don’t rely on the default update funtionality in WordPress. Hook into it and give users a step by step form to complete inorder for the upgrade to be made. You know what version they are upgrading from and can adjust accordingly. You can also present a screen with the changes before upgrading that they need to accept. This of course relies on you having added these capabilties to your plugin before hand.

In conclusion

The main thing is to write tests for your code and try not to hardcode things. Remaining backwards capable and having a good structure in your code goes, from my perspective, hand in hand. If you have good structure you can also make bigger changes if the need arises. Writing more complex functionality sometimes require you to do some thinking before you get going. Especially if you write code that you think people will expand upon it with new functionality.

If you have comments or other ideas on the subject of backwards capability and code structure please leave a comment I would love to learn more about it. If you think my thoughts on the subject is completly bonkers I don’t mind hearing about that as well. Just try to do the shredding as politely as possible.