Debugging techniques for Web development 2 – Configuring your environment

In the previous issue we discussed using the right tools to more easily debug your Web development scripts.

Now these tools can be made even more useful if they are configured in a way that makes the developer’s life easier.

2. Configuring your environment for better productivity

If you don’t make yourself at home in your own code base, who will?

Configuring your environment can mean many things, but the points I would like to cover in this post are:

  • Setting up your editor / subversion / server so that you can get through the day in the most painless way
  • Creating a development environment that works alongside your production
  • Writing your code so that if something goes wrong you can easily pinpoint the cause and quickly fix it

Setting up editor / server & co

When you start working on a project, you often want to start small and avoid wasting massive investments to organise your code. Now, once you’ve done it a few times, you’ll realize that settings up a clean and comfy environment doesn’t take that much effort, and really pays off in the long run.

By that I mean the following:

On the editor

  • Making sure your editor is the right one for the language you use, and is extendible. This means it allows you to define keyboard shortcuts for the kind of things you often do. I like creating shortcuts for the following stuff:
    • firing up a svn diff or svn commit command. Most of the time it’s just to figure out if I’ve changed the file at all, sometimes it’s to do an actual commit
    • opening the browser at a given location
    • bookmarks for often-used config files (e.g. the apache ini)

On the server

A common practice with beginners using a webserver is to put all their projects under a single root, so they can then get pages at

http://localhost/project1 (which will point to c:\bla\bla\root\project1)
http://localhost/project2 (which will point to c:\bla\bla\root\project2)

Now, by tweaking some config, you can achieve a much more symmetric setup, where your 2 projects will be hosted on the root of some (virtual) server. In order to do so, do the 2 following things:

  • Add as many entries as you like in your hosts files (see Wikipedia), of the following form
# all these names point to your localhost
# but Apache will see the difference
# due to the VirtualHosts directive

127.0.0.1    project1
127.0.0.1    project2
  • Add virtual hosts in your Apache config, like this

    DocumentRoot "c:\bla\bla\root\project1"
    ServerName project1

The result of that is the ability to get the website (from your computer that is!) at an address like

     http://project1/

I know – in theory it shouldn’t matter whether your pages are at the root of your server or not! However in practice it is a source of so much pain and wasted time when your local setup runs on a sub-folder of the root and your production setup runs on the root, and some things do not work as expected on your production server. Life’s too short to try and make it work (and some of the frameworks are just rubbish at working outside of the root of a server and you do not plan to fix them).

In the database

Not that I am doing it routinely, but some cool database features can be leveraged to make your life easier:

  • Views: these are queries that you can use as if they were a table. So for instance, if you’ve got a query like this
SELECT user_id, user_name FROM users WHERE user_type = 'admin'

Then you would create a view by doing this

CREATE VIEW admins AS
SELECT user_id, user_name FROM users WHERE user_type = 'admin'

Subsequently, you can then do

SELECT user_name FROM admins

This can be useful when you are debugging and don’t want to endlessly retype the same WHERE clauses.

  • Triggers: never used these, but they look pretty powerful. They allow you to have cascading effects in your tables, or prevent you from doing operations which would violate relationships (e.g. preventing you from deleting a user in the users table if the posts table still has entries linked to that user)

In your system

Create scripts to perform standard tasks.  I personally think that people who write batch / unix scripts are masochists, and so I do it in python much more easily. That includes

  • Issuing ssh commands to a remote server (e.g. svn update)
  • Tranfering files through SCP
  • Creating a dump of your mysql database using mysqldump
  • Going to some of your favorites directories

Creating a development environment

For some people, a development environment is something they are not even aware of and which consists of a series of commented print (PHP) or console.log (Javascript) statements scattered randomly in their code, and which will be uncommented when needed – and sometimes accidentally pushed to production with unpredictable consequences…

Any serious framework will consider the development environment an integral part of the project, and provide an API that will allow the developper to leave trails of what is happening. At the same time it is important that the user does not see any of this, be it for better customer experience or sometimes as a security precaution (i.e. avoiding to give the user insights in the server-side code). This can happen by having the notion of environment at the API level. This means that

  • At the root of your code (the HTML page for Javascript, your controller for PHP), there needs to be an assesment of what the current environment is, i.e. ‘development’, ‘testing’, or ‘production’. This can be done by some environment variable, some local file, can be based on the hostname (or the virtual host – even better!), or perhaps by some cookie on the user’s browser.
if ( /*... */ )
    $ENVIRONMENT = 'production'
if ( /* ... */ )
    $ENVIRONMENT = 'development'
  • Any setting that has reasons to vary depending on the environment should be predicated on the environment. So, for instance, the MySQL database could be a function of the environment, for instance like this
$db['production']['database'] = 'projectdb_prod';
$db['dev']['database']        = 'projectdb_dev';

And the subsequent calls will be like this

$server = $db[ $ENVIRONMENT ]['database']

A golden rule – A SINGLE CODEBASE

 The ‘single codebase’ rule means that you do not want to have different versions of the same file depending on the server. This would be difficult to maintain / deploy (not least because it would have to be outside of your control version system), and soon or later you are going to accidentally overwrite the config of one server onto another and scratch your head to understand the source of a bug. Instead, you want all the settings to be in the common codebase, and ensure that the choices are done on-the-fly depending on the environment / host / OS.

Writing your code so it is debuggable

Separation of concerns

Modularity, for example as in MVC frameworks can dramatically improve the debuggability of your code. This is because the business logic will be testable independently of the GUI (which tends to be tough to test).

Hard to give precise guidelines, but let’s say that the rule of thumb is to make it so that you can test key features of your application with one-liners and without having to use the user interface.

Using configuration settings or even bits of code that can be changed at runtime

A technique I have found useful, which is more related to fine-tuning the Javascript part of your application than debugging it, is that of using configuration settings.

For instance, in an app I am developping, I need to parametrize many settings so that some graphics looks as good as possible.

Previously my code was looking a bit like this

some_setting       = 0.5
some_other_setting = 25
timestep           = 0.25
some_curve         = timestep * some_setting + x * some_other_setting + /* .. */

This meant that, in order to test changes to any of these settings (say, move some_setting to 0.6), I needed to reload my page. Given that I am now convinced that one-page (javascript) apps are the future of the web, I end up with a page whose initialization can take up to 10 seconds. Given the features, this is acceptable for the end-user, but for debugging purposes, it is a bit of a pain.

The idea I came up with (well this must be a design pattern – I just don’t know how it’s called) was to use a global function to get and set settings, and attach a callback that gets triggered every time the settings are changed.

This broadly worked as follows.

In a library, I would have

// ideally this first line should be in a project specific library whilst
// the rest is totally generic
function DefaultSettings() { return { 'SOME_SETTING' : 0.5 /* ... */ } }
current  = DefaultSettings()
onchange = function() {}
function Settings(key) { return current[key] }
function SettingsSet(key,value)
{
     current[key] = value;
     onchange(); // trigger callback
}
function SettingsReset() { current = DefaultSettings(); onchange(); }
function SettingsAttachCB( callback ) { onchange = callback }

Then my code above would become

some_setting       = Settings('SOME_SETTING');
some_other_setting = Settings('SOME_OTHER_SETTING');
timestep           = Settings('TIMESTEP');
some_curve         = timestep * some_setting + x * some_other_setting + /* .. */

And in my initialization function, I would have a line like this

SettingsAttachCB( SomeRefreshFunction );

This would mean that, once the page has loaded, I can go to the Chrome debugger console, type SettingsSet('SOME_SETTING', 0.6), and that will automatically trigger SomeRefreshFunction.

Settings

As a matter of fact, I even wrote a little HTML widget that turns current in JSON into a popup window, and allows some users to modify it.

Continue reading

Debugging techniques for Web development 1 – Using the right tools

Over the years I’ve become a bit of a debugging freak.

I wish I was able to write code that works straightaway, but for some reason, I just love programming by trial and error. So I go through many cycles before it’s just right, and without some ways to shorten these cycles, programming would be a nightmare for me.

So I would like to share a few techniques gathered over time, which hopefully will help you as is, or will inspire you to develop a debugger’s mindset. Be it in Web programming or any scripting environment.

The techniques will be explained in several blog posts, which will be about

  1. Using the right tools – to do complex things in a simple way rather than the other way round…
  2. Configuring your environment for better productivity – if you don’t make yourself at home in your own codebase, who will?
  3. On-the-fly coding instead of edit/reload/edit/reload – coz’ life is to short to spend it waiting for your script to reinitialize.

Let’s then start with the first piece of advice, which I would assume is the easiest to apply

1. Using the right tools

To do complex things in a simple way rather than the other way round…

In the past, I’ve sometimes been doing stuff in a convoluted fashion because I was not aware that certain tools existed, which could have made things so much easier. This is a general principle, and so it’s mostly a question of discussion with the community to figure out which tools are going to help you in your particular project.

Let me however give a few tips specific for Web development.

Installing a [L/W]AMP stack

Unless you’re just writing 2-3 web pages, please do not do Web development by loading the files as is. Put yourself in an environment similar to the final deployment stage, by running all the pages through a proper webserver.

One reason for that is that Ajax call will likely not work from local pages due to the X-domain policy. You may also face issues of links working in your development environment but not on the server, or vice versa. And of course, PHP scripts and .htaccess stuff (i.e. Apache configs per directory) will definitely not run if you call them directly rather than through a webserver…

So – on Linux systems, the webserver  is usually provided out of the box by LAMP (Linux/Apache/MySQL/PHP), and on windows, you can install EasyPHP which provides these with minimal hassle.

Javascript

I really fell in love with JS the day I discovered the Chrome debugger. Obviously it requires using Chrome, which I fail to see a reason to do without – see table on the side (source: Wikipedia).

ScreenClip3

The debugger is invoked by pressing CTRL-SH-J, and will appear at the bottom of the page. It can be detached as a separate window, which can be desirable if your HTML layout is involved.

The debugger features a set of terrific features – see the Chrome Developer Tools page for more details.

One of them is the console (press ESC to open/close it, whilst the debugger is on). The console allows you to evaluate expressions on the fly after the page has loaded. This is extremely useful if you want to invoke functions defined in your script, or modify the state of global variables . It is also very cool for CSS experiments (see CSS debugging paragraph below).

The console is also terrific for leaving trails of your code’s execution using console.log, without impacting the HTML layout, like this

ScreenClip

As for modifying variables, this goes beyond just  numbers / strings. In a future issue I intend to discuss that more in detail, especially the use of the console to modify and test callbacks / event functions without having to reload the page. As a preview, let me mention the use of the console to modify DOM objects.

Say, for instance that you are trying to make your website look better by changing the css. Reloading your page might be fine if it’s a lightweight one, but as soon as your project starts growing, your main page is bound to load tons of stuff at startup. This might make it a pain to fine tune styles / colors / dimensions.

Assuming you are using JQuery, you can easily change attributes using the Chrome debugger. There are 2 methods

  • go on the part of the page you want to modify, right-click and choose ‘Inspect element’. This will open up the debugger, and will open a window on the right with all css attributes. You can modify any of these manually.
  • my preferred method is to go to the console, and do something like this
$("#someid").css( { width : '10px', color : 'white' } )
$("#somepopup").show().css( /* ... */ )

This will modify the object on-the-fly. Once you’ve sorted out your settings, you can integrate it in your script for the next time you reload your page.

PHP

The more advanced you are in PHP, the more likely you are to use frameworks which tend to provide extensive debugging information. One of these is Symfony, which I find very impressive, but personally way over-the-top for my usage. I therefore limit myself to CodeIgniter + Datamapper, which are fine, but do not have embedded debugging features.

However – debugging is made extremely simple using Xdebug. It is an apache extension that is part of the Zend framework, but can be installed on its own. In my case, it came bundled with EasyPHP, and it turned out that my editor Notepad++ has a simple plugin DBGp which communicates with Xdebug and allows debugging PHP scripts as easily as you would do it in Javascript in the Chrome debugger.

The setup consists of downloading Notepad++ and the DBGp plugin, and moving the plugin to the right Notepad++ folder. The Notepad++ settings require you to specify the server (typically this would be localhost), and turn the debugger on by going to the ‘DBGp / debugger’ menu, which will make the debugger appear in the bottom part of Notepad++.

On the server side you will need to enable Xdebug, which is done in the php.ini with something that in my case was this

zend_extension = "${path}\php\php546x130302144954\php_xdebug-2.2.1-5.4-vc9.dll"
xdebug.default_enable=0
xdebug.remote_enable=1
xdebug.remote_host=127.0.0.1
xdebug.remote_port=9000
xdebug.remote_handler=dbgp
xdebug.remote_mode=req
xdebug.remote_autostart = false

What then happens is that every time you load your page (i.e. by going to http://localhost/yourpage), it will invoke the debugger on Notepad++, which will start blinking. You will need to put breakpoints by pressing CTRL-F9 on a line of your scripts where you want to start stepping, and you can step the usual way (using the little buttons at the bottom left of the Notepad++ screen). It should look like this

ScreenClip2

MySQL

Working with databases can be a total nightmare without the proper tools. You need to be able to test your queries independently of your code, and I would recommend installing PHPMyAdmin, which comes bundled with EasyPHP or is easily installed on a LAMP stack (it’s all PHP based so requires no shell access to the server). Given that it is accessible through a web browser, this allows you to do basic tests or SQL operations from anywhere. It has a visual interface for many operations, including data modification, table creation, and even db synchronization.

A more powerful tool is MySQL workbench. The drawback is that it requires local installation on Windows, but the interface is less sluggish than what you would except from a web-based interface such as PHPMyAdmin.

Control versioning

Control versioning software allows you to track changes in software you are writing, by having the developer ‘committing’ changes when he deems they are worthy of being used in production. This then allows him to see differences in different versions of his scripts, which often helps to pinpoint the location of freshly introduced bugs.

Working on even small size projects without control versioning is a bit like driving with your eyes closed. If everything runs fine and you’re the only one on the road, it’s not an issue, but if there is any unexpected event, you’re heading for a crash.

Anyhow – installing control versioning is so simple that there is no excuse to do without.

I know git’s the most fashionable tool nowadays, but I happen to be using subversion which I find just fine.

The setup goes as follows

Server side – 2 possibilities (ok this might be confusing if the server is on your own machine)

  • Creating a local repository on your computer or any computer on your network. This is as simple as typing svnadmin create 'your_repository', then launching the server by doing svnserve -d -r 'c:\path\to\your_repository'.
  • Creating an account at CloudForge, for instance, which will host your code for free (but it is limited to a single user).

Client side

Once the server is running, you will need a client to access it, and modify the files. I like TortoiseSVN, which allows you to perform all kinds of operations from the Windows explorer.

The benefit of using a CVS system in a live environment such as a website is that you can do without FTP or SCP, and apply/rollback changes at a whim. In my case, I setup a batch script to ‘svn update’ my production server (through ssh). This means I can upgrade my production server without having to figure out what needs update or what not.

This does not prevent me from keeping external files (images / libraries) in separate directories which are ftp’ed once and for all, and can even be shared across different versions of my website by using (Linux filesystem) links.

Get a proper server

OK so unlike the other points, this one is not that easy to apply. People may not have the budget for an expensive setup, but I highly recommend getting server space with a shell access. I discovered Rackspace, which costs me 14£/month, for which I have a 512Mb ubuntu server with 20Gb RAM and root access. It took me less than an hour to install AMP / subversion, check out my repository, sync my db and have my website online as on my laptop. Just installing Ubuntu on a computer can sometimes take 45 minutes, whilst Rackspace provide preconfigured images which get installed in a minute.

Of course, you can do without, but having no shell access may mean that you can not use subversion to update your website, and you are at the mercy of any discrepancy between your home setup and the server (which at a typical provider will have an older version of PHP rather than a newer, resulting in some language idoms working at home and not on the server).

Given that you have root access, nothing refrains you from setting up several accounts and sharing the server with friends. Because of the root access, you can do the following

  • setup a subversion server for free, which would probably cost you 5-10£/month if you went  for the premium solutions at Cloudforge, for instance.
  • setup an unlimited number of domains hosted on your server (Rackspace provide you with a control panel to edit DNS entries)
  • install a nodejs server, or Django, or upgrade to whichever version you want / need

Conclusion

Depending on your experience and your luck googling for some of the tools above, you may have things working out-of-the-box, or you may struggle and give up and miss out on some terrific productivity boosters.

Don’t hesitate to ask for help if you are facing issues!