Posts Tagged ‘symfony’

Writing functional tests for HTTP auth secured actions

Wednesday, September 2nd, 2009

Yesterday i was forced to write a functional test for a HTTP auth secured action. Pretty easy i thought, but it took me hours for finding the correct lines.
The HTTP auth mechanism works like it is explained in the blog post HTTP authentication with sfGuard, although the sfGuard plugin is not used in our application.
This secure action takes the PHP_AUTH_USER and PHP_AUTH_PW $_SERVER variable and validates it. If the validation passes, a redirect to the called uri is executed.

How does a functional test look like?

Thanks to the setAuth method of the browser test class, we can simulate an incoming request with HTTP auth data pretty easy. This authentication will last for the very next request, subsequent requests have to be authenticated again.
After calling the setAuth method, we can call some action as usual.

1
2
3
4
$browser->setAuth($username, $password)->
 
// call an HTTP Auth secured action
get( '/some_module/some_action' );

Now we have reached the tricky part. Due to the secured action, the request is first of all forwarded to the defined secure action, here it is the login action of the auth module. Within this action, a redirect is called to the original uri.
This means, that the next simple test code will fail:

1
2
3
4
5
6
7
8
9
$browser->setAuth($username, $password)->
 
// call an HTTP Auth secured action
get( '/some_module/some_action' )->
 
with('request')->begin()->
isParameter('module', 'some_module')->
isParameter('action', 'some_action')->
end()->

The current module is not the original called some_module one, but the auth one. This process could be tested with the isForwardedTo method.
If we would check the response at this point, we won’t get the correct response, but a redirection statement:

1
<html><head><meta http-equiv="refresh" content="..."/></head></html>

Handling this redirect, needs two more method calls in the test code. After adding the methods isRedirected and followRedirect we have everything we need to provide a good test for this process.

The complete test code looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$browser->setAuth($username, $password)->
 
// call an HTTP Auth secured action
get('/some_module/some_action')->
 
// due to our used HTTP auth, the request is redirected to our secure module
isForwardedTo('auth', 'login')->
 
// it redirects back to the original uri
isRedirected(true)->
 
// we have to follow the redirect, otherwise we do not get
// the correct response
followRedirect()->
 
// did we recieved the correct response?
with('request')->begin()->
isParameter('module', 'some_module')->
isParameter('action', 'some_action')->
end()->
 
responseContains('foobar');

Worker classes in symfony – Part 2

Thursday, June 11th, 2009

This is the second and final part of my series about worker classes in symfony.
The first part has shown, how worker classes could make development easier, if the main logic of your project is moved there. This article is focused on the scalability fact of those classes.
Let’s switch from the newsletter example from the first article to another one, which may show up the scalability advantage better: fetching an user object by its primary key/id.

Jumping back to an normal action, it could look like this (using Propel):

1
2
3
4
5
6
7
8
class userActions extends sfActions
{
  public function executeProfile(sfWebRequest $request)
  {
    $this->user = UserPeer::retrieveByPK($request->getParameter('id'));
    $this->forward404Unless($this->user);
  }
}

This is quite simple and should look familiar to you. Note that i ignored the possibility to fetch the user object directly from the routing, otherwise it would be hard to explain the next steps.

There are two problems we are faced to, if we want to scale this action. The first one is the restriction to Propel query code (here it is: UserPeer::retrieveByPK). Switching to another ORM requires a lot of time, while browsing through the complete project and replacing ORM-specific code.
The second one is the returned model class itself. The model defines no clear interface to its accessors. Getters, setters or attributes will be added and droped if you change something in its according schema. It is even more work, when you have to adept the code provided by the model class before you can switch to another ORM.
In fact, it would be quite easier to change from Propel to Doctrine, but only imagine it the other way round…
I am also aware of the existance of the DBFinderPlugin, which provides a common interface for both Propel and Doctrine. But what will happen, if you do not use one of those ORMs? Yes, it would not work and therefore we could not use it either.

Ok, how would the code of the first listing look like using a worker class?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
class UserWorker extends sfBaseWorker
{
  /**
  * Returns an hydrated user object by user id
  *
  * @param $int id the user id
  *
  * @throws UserWorkerException
  *
  * @return User
  */
  public function getById($id)
  {
    if(!$user = UserPeer::retrieveByPK($id))
	{
		throw new UserWorkerException(sprintf('User with id %s could not be fetched', $id));
	}
 
	return $user;
  }
}
 
class userActions extends sfActions
{
  public function executeProfile(sfWebRequest $request)
  {
    $worker = new UserWorker($this->dispatcher, $this->context);
    $this->user = $worker->getById($request->getParameter('id'));
    $this->forward404Unless($this->user);
  }
}

Did we solved our two problems now? Unfortunately not yet completely. Our action is freed from ORM-specific query code, but it still processes with the returned user model class.

The solution
We solve the problem, when we introduce proxy classes and well defined interfaces for our models. Sounds complex? It might be :). Let’s see some code, here we go:

A very simple and shortend interface for the user model could be:

1
2
3
4
5
6
7
8
9
10
interface UserInterface
{
  public function getId();
 
  public function getName();
 
  public function setName($v);
 
  public function save();
}

The implementing proxy class for the user model:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
class UserProxy implements UserInterface
{
 
  private $user = null;
 
  public function __construct(User $user)
  {
    $this->user = $user;    
  }
 
  public function getId()
  {
    return $this->user->getId();
  }
 
  public function getName()
  {
    return $this->user->getName();
  }
 
  public function setName($v)
  {
    return $this->user->setName($v);
  }
 
  public function save()
  {
    return $this->user->save();
  }
}

Adding this proxy class to our user worker:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class UserWorker extends sfBaseWorker
{
  /**
  * Returns an hydrated proxy user object by user id
  *
  * @param $int id the user id
  *
  * @throws UserWorkerException
  *
  * @return UserProxy
  */
  public function getById($id)
  {
    if(!$user = UserPeer::retrieveByPK($id))
    {
	throw new UserWorkerException(sprintf('User with id %s could not be fetched', $id));
    }
 
    return new UserProxy($user);
  }
}

Now we are done. The action (or template) does not process anymore with a concrete ORM model. With the help of the proxy class, we can change the underlying model layer without breaking anything in the action. By providing a well defined interface, the proxy class manages how a model could be modified and accessed.

At this point, you should understand the reason for all those changes. It may look like a huge overhead for just retrieving some data out of the model. But in our case it was a precondition for our global architecture.
Our models and our main business logic is partly served by and processed in SOA applications (there we use Propel and i hope in some near days also Doctrine). I implemented a mechanism, which allows worker classes to access those SOA applications very easily. They are fetching the serialized data from those applications, hydrate them into proxy classes and provide them to the rest of the project.

I could not show the original code, but it looks similar to this (it differs in little parts from the former listings):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
class UserWorker extends sfBaseWorker
{
  /**
  * Returns an hydrated proxy user object by user id
  *
  * @param $int id the user id
  *
  * @throws UserWorkerException
  *
  * @return UserProxy
  */
  public function getById($id)
  {
    $soaQuery = new SoaQuery();
    $soaQuery->setAction('user', 'getById');
    $soaQuery->setParams(array('id' => $id));
 
    if(!$serializedData = $soaQuery->execute())
    {
	throw new UserWorkerException('some soa error ....');
    }
 
    $proxy = new UserProxy();
    $proxy->hydrate($serializedData);
    return $proxy;
  }
}

In this way, action code is really short, it is completely decoupled from the logic how the model is originally handled and it works quite well. If we decide to change something in our architecture some day, we only have to change our worker and proxy classes. The rest of the project will not notice those changes, as it should be in a layered architecture.

Sure, this step is nothing for small applications. But when your project grows and grows, the number of requests in a second turns from dozens to hundreds, then you’ll be thankful, if your project provides this approach.

Worker classes in symfony – Part 1

Wednesday, June 10th, 2009

This article is the first of a two-parted series about worker classes in symfony. It will describe the problems a developer may be forced to, when he has to execute some “action” code, called from several positions in a project.
The second upcoming article will show up the advantages of worker classes reflected to scalability and maintenance costs, the main advantages, which changed my mind :)

The division in actions and components of your business logic is a good way for reusing your business logic.
But did you ever reach a situation, where you had to execute the same code from an action and a component?

Let’s take an example for demonstrating this fact better.

An action takes an email and should add it to a newsletter system. This newsletter system adds this email to the database if it does not exist already and sends an approval email to its owner.

The standard code within an action could like this (form handling is not displayed):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class emailActions extends sfActions
{
  public function executeAdd(sfWebRequest $request)
  {
    $email = $request->getParameter('email');
    $entity = NewsletterEmailPeer::getOrCreateByEmail($email);
	if($entity->isNew())
	{
	  $this->sendApprovalEmail($entity)
	}
  }
 
  public function sendApprovalEmail($entity)
  {
    $mailer = new SomeMailer();
    $mailer->setSubject('some subject');
    $mailer->setBody('blah');
    $mailer->addReciever($entity->getEmail());
    $mailer->send();
  }
}

No problem when there is only one point on your website, where a user can add his email to the newsletter. But imagine an according newsletter field may be listed on an editing profile form or a contact form and this listed code should be executed there too after submitting the form. Some days ago, i was forced to execute some similar code in a component (don’t ask why) and a action. I didn’t find a nice solution first.

One could move this code completely in a component and call it from several actions, but one could only fetch a complete rendered component. There is no way (or?) to execute a component and get something different returned than rendered markup.

At this point you have three possibilities:
1. forward to the email action
This sounds like a hack and the user may be wondering why a newsletter result page is displayed, after he did alter his profile etc. Nasty :P

2. Call the extra logic directly in your model
One may decide to move this logic directly to the model.

It could look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class NewsletterEmailPeer extends BaseNewsletterEmailPeer
{
  static public function addEmail()
  {
	$entity = self::getOrCreateByEmail($email);
	if($entity->isNew())
	{
	  $entity->sendApprovalEmail($entity);
	}
  }
}
 
class NewsletterEmail extends BaseNewsletterEmail
{
  public function sendApprovalEmail($entity)
  {
    $mailer = new SomeMailer();
    $mailer->setSubject('some subject');
    $mailer->setBody('blah');
    $mailer->addReciever($entity->getEmail());
    $mailer->send();
  }
}

If you prefer this possibility you should reconsider the original definition/usage of data models:

a data model is an abstract model that describes how data is represented and accessed

(Quote from wikipedia)

That’s it. There is no reason to put more logic into models than modifing and saving data. Therefore, this possibility should not be implemented at all!
However browsing through symfony plugins reveals quite fast some files where this possibility is realized. I do not like this approach and i hope that less stuff will be implemented in this way in future days. Feel free to leave me a comment, when you prefer this solution yet and try to convince me of this handling.

3. Create a new class for this logic
The third and last possibility is to create a new class and place this additional code in it. Those classes are “worker” classes, which do the hard part of your system. Actions, components, tasks etc. should access those workers when some main logic should be executed. This is my prefered solution and you should see quite fast why.

Stepping back to our example, the original action could like this:

1
2
3
4
5
6
7
8
9
class emailActions extends sfActions
{
  public function executeAdd(sfWebRequest $request)
  {
    $email = $request->getParameter('email');
 
	$worker = new NewsletterWorker($this->dispatcher, $this->context);
	$worker->addEmail($email);
}

Wohoo, quite short. Isn’t it?

The according worker class content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class NewsletterWorker extends sfBaseWorker
{
  public function addEmail($email)
  {
    $entity = NewsletterEmailPeer::getOrCreateByEmail($email);
	if($entity->isNew())
	{
	  $this->sendApprovalEmail($entity)
	}
	return true;
  }
 
  private function sendApprovalEmail($entity)
  {
    $mailer = new SomeMailer();
    $mailer->setSubject('some subject');
    $mailer->setBody('blah');
    $mailer->addReciever($entity->getEmail());
    $mailer->send();
  }
}

Note: Save the worker classes in the according module’s lib dir, if you do not want to break the modular structure. But no one will cry, if you create a global dir in your project lib folder for them, like it is done for the models.

Advantages of worker classes

  1. actions could be shrinked huge in size
  2. worker classes are more decoupled from the rest of symfony and could be tested better
  3. you can replace the content of an worker method without breaking actions
  4. more in the second article

One main advantage was not mentioned so far: scalability. Read in the second and still upcoming article about this advantage and why i decided to introduce those worker classes in my current project finally.

memcached as singleton in symfony

Friday, June 5th, 2009

If you have to deal with performance issues, you should be used to Memcached. Memcached is a really great tool to setup a quick and easy to use caching mechanism. Memcached could be used for everything, caching your database queries, caching complete rendered HTML pages.. everything.

I was forced to integrate memcached in a new symfony 1.2 project and i wanted to use only one memcached instance across my complete project. Let me share the steps i have done.

Symfony provides the sfMemcacheCache class since 1.1. With this class you are able to change your view caching to memcached and drop the default file caching. Thanks to the great factory mechanism it is no big deal to switch the caching class.
I cleared the cache and changed the view_cache part of my factories.yml.

1
2
3
4
5
6
7
8
9
10
11
#factories.yml
dev:
  view_cache:
    class: sfMemcacheCache
    param:
      lifetime:                  86400
      prefix:                    some_prefix
      servers:
        server_01:
          host: some.host
          post: 11211

Afterwards, i returned to the website and noticed a way faster rendering of cached pages.
Ok, this was the easy part. But what about using memcached in your project besides view caching and you only want to have one instance be available?
I needed some time to figure out a nice solution (maybe it’s a hack, but i couldn’t think of a better one):

  1. subclass sfMemcacheCache and overwrite the initialize method
  2. create a new class which provides memcached as singleton

Step 1: Subclass sfMemcacheCache and overwrite the initialize method
sfMemcacheCache provides a possibility to pass an existing memcache instance in the options array of its initialization method (via the memcache option key). I took this possibility, created a child class, build a valid memcached instance in the overwritten initialize method and passed it to the parent method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class sfMemcacheSingletonCache extends sfMemcacheCache
{
  /**
   *
   * @see sfMemcacheCache#initialize()
   */
  public function initialize($options = array())
  {
    // we create our own memcache instance here
    // sfMemcacheCache checks for a memcache key in the
    // options array and takes this value without
    // creating a new memcache instance
    // other option values, like prefix etc. are assigned to the subclass
    $memcache = sfMemcache::getInstance();
 
    $options['memcache'] = $memcache;
    parent::initialize($options);
  }
}

Warning:
Clearing the symfony cache will trigger flushing your complete memcache. If you do not want to allow this, you may overwrite the clean method in this child class

1
2
3
4
  public function clean($mode = sfCache::ALL)
  {
    // do nothing
  }

First i left the connection settings in the factories.yml. But i recognized soon, that i will not be able to create a valid memcached instance, when an environment has caching disabled.
Therefore, i decided to move the connection settings in the app.yml file and accessed the according options via the normal sfConfig class (you will see this in Step 2).

The factories.yml looks afterwards like this

1
2
3
4
5
6
7
#factories.yml
dev:
  view_cache:
    class: sfMemcacheSingletonCache
    param:
      lifetime:                  86400
      prefix:                    some_prefix

and the app.yml like this

1
2
3
4
5
6
7
8
all:
  foo: bar
  .array:
    memcache:
      servers:
        server_01:
          host: some.host
          port: 11211

Step 2: Create a new class which provides memcached as singleton
The singleton class is already listed in the code snippet of Step 1. Here are the internals (it’s a normal singleton class with quite more methods as shown here):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
class sfMemcache extends Memcache
{
  static private $instance = null;
 
  private function __construct()
  {
    $this->initialize();
  }
 
  static public function getInstance()
  {
    if(!self::$instance)
    {
      self::$instance = new sfMemcache();
    }
 
    return self::$instance;
  }
 
  private function initialize()
  {
    $this->options = sfConfig::get('app_memcache', array());
 
    // START: taken from sfMemcacheCache::initialize
    if ($this->getOption('servers'))
    {
      foreach ($this->getOption('servers') as $server)
      {
        $port = isset($server['port']) ? $server['port'] : 11211;
        if (!$this->addServer($server['host'], $port, isset($server['persistent']) ? $server['persistent'] : true))
        {
          throw new sfInitializationException(sprintf('Unable to connect to the memcache server (%s:%s).', $server['host'], $port));
        }
      }
    }
    else
    {
      $method = $this->getOption('persistent', true) ? 'pconnect' : 'connect';
      if (!$this->$method($this->getOption('host', 'localhost'), $this->getOption('port', 11211), $this->getOption('timeout', 1)))
      {
        throw new sfInitializationException(sprintf('Unable to connect to the memcache server (%s:%s).', $this->getOption('host', 'localhost'), $this->getOption('port', 11211)));
      }
    }
    // END: taken from sfMemcacheCache::initialize
  }
 
  protected function getOption($name, $default = null)
  {
    return isset($this->options[$name]) ? $this->options[$name] : $default;
  }
}

Note: The core of the initialize method is taken from the original sfMemcacheCache class.

The most important custom part is found in this line:

22
 $this->options = sfConfig::get( 'app_memcache', array() );

The options for the memached connection is saved in the app.yml and not in the factories.yml. This is needed – like i already mentioned it in Step 1 – to be able to create from everywhere of the code the memcached singleton, independant to the caching setting.

That’s it. Now, I was able to use only one instance of memcached in my view caching and my custom caching stuff.

Maybe this helps someone, who has the same requirement as me.