Tim's Software Development Blog

Abstract methods in Python

2011-05-31T16:07:00.012+03:00

Python has no concept of an "abstract method" on a language level. This makes developers often implement abstract methods incorrectly. Even though there is no language level construct for this - there is a pretty strict and well known pattern for abstract methods, and there are according means for that. Hereafter is a correct way to implement it (and most often - the only correct way). I hope this post may be useful for new Python developers who are not familiar with the "abstract method" concept in Python yet.

Problem:
Implement an abstract method. In most simple terms - abstract method is a method with a defined signature, and with no implementation; if class A defines an abstract method, it is supposed that classes inherited from A should implement it (not necessarily direct subclasses, but some subclass should or otherwise what's the point of it?). Thus, abstract methods are used to define a behaviour this or other classes use, while the implementation of this behaviour is provided by subclasses. Most often this is used to define interfaces or implement a "factory method" pattern.

Common error:
Most common error is to merely omit an abstract method declaration, use the not declared method whenever needed, and suppose subclasses actually add it to the class. For example, the following class uses method connect(), which is defined in a subclass only.


class ConnectionPool(object):
  def __init__(self):
    self.pool = []
 
  def hasSpareConnection(self):
    # here goes the method implementation
    # I don't put here for simplicity

  def acquire(self):
    if not self.hasSpareConnection():
      connection = self.connect()
      self.pool.append(connection)
      return connection

class MySqlConnectionPool(ConnectionPool):
  def connect(self):
    # connection = connect to MySQL somehow 
    return connection

This is not correct for a number of reasons. First of all - how would a subclass author (unless it is you for the eternity and you have an ideal memory) know that he or she needs to implement it? It is not defined and not documented anywhere. Secondly, classes using these "not declared abstract methods" are not readable at all: when you read it not having a subclass - you see a method being executed but can't figure out where is it defined. Finally, you get an AttributeError with a weird message like "attribute 'bla' is not defined", which is true, but this message proposes no solution for a subclass author.

All these issues are solved in a next section: the method is always defined there in a super class, its docstring describes what it does, and the error message says how a subclass should behave.

Solution 1:
Python has no language level construct for abstract methods, but you force class users to implement a method by throwing a NotImplementedError from a method you define as an abstract. Ideally you should as well put a docstring for the method and provide a detailed error message.


class ConnectionPool(object):
  # ...
  def acquire(self):
    if not self.hasSpareConnection():
      connection = self.connect()
      self.pool.append(connection)
      return connection

  def connect(self):
    '''Creates a new connection to the DB 
    and returns it. Should be implemented 
    in a subclass'''

    message = "Method connect() should be \
    implemented and return a newly \
    instantiated connection to the DB"

    raise NotImplementedError(message)


class MySqlConnectionPool(ConnectionPool):
  def connect(self):
    # connection = connect to MySQL somehow 
    return connection

Now suppose you implement new OracleConnectionPool. You create a subclass, don't implement a connect() method, and run a program. What you get is an error message: "Method connect() should be implemented and return a newly instantiated connection to the DB". Now you have plan! Go on, implement it and your job is done! This is much easier than digging into the AttributeError trace back and the base class implementation to find out what's happening there.

Solution 2:
There is another (to my mind - better) solution ever since Python 2.6 - it's an abc modue. I don't think it is worth to describe it in all details here given that its documentation is pretty complete and easy to understand. I will only show an example of its usage in this concrete example, so that you can compare it with the previous one. Among its benefits I will mention the only one I personally consider the most important - users won't be able to instantiate an abstract class or instantiate its ancestor unless this ancestor implements all abstract methods. Yes, class users will see a standard error message once they try to instantiate a class, not only when an abstract method is executed.

Changes are made to the ConnectionPool class only. We will do the two things: first, delcare it as an "abstract base class", and define certain methods as abstract. Ancestror classes will look same way as with the previous solution - just implement abstract methods and you're good to go.

from abc import ABCMeta, abstractmethod

class ConnectionPool(object):

 __metaclass__ = ABCMeta

  # ...
  def acquire(self):
    if not self.hasSpareConnection():
      connection = self.connect()
      self.pool.append(connection)
      return connection

  @abstractmethod

  def connect(self):
    '''Creates a new connection to the DB 
    and returns it. Should be implemented 
    in a subclass'''

    return None

Note: many thanks to user 'naglfar' for the hint about "abc" module. See comments below for details.

Resume:
Well, I'm used to put a "resume" to all my posts, but this one is pretty self-descriptive - please, consider this pattern when implementing abstract factory methods or interfaces. It is a well known pattern and most Python developers actually expect your class to behave this way if you suppose subclasses should implement its abstract methods.

Disclaimer: the example above is imaginary and is not likely how you implement a connection pool indeed.

Add custom events to your GWT widget or, "Rich TextEditor, part 2"

2010-06-29T22:47:00.021+03:00

This post is a "part 2" of the "Create a full-fledged GWT Rich Text Editor widget". It is not necessary though that you read previous one, to dive into here. In previous post I described how to create your own TextEditor GWT widget by wrapping the TinyMCE Java Script library. In this article I will continue to enhance the widget and will add a support for a custom event - "save" event in particular.

You may download the source code. This is also good news for those who read the previous post - I didn't attach the source code then, but was asked for later, so here it is. In fact, this download is a complete version of the widget and it also includes the changes described in this post.

Well, let's start...

Design the event handling from user standpoint

What the widget lacks is an ability to save a document being edited. Without this ability its usage is limited to embedding it to certain form, having its own submit button. In fact, there is no even a "Save" button in a widget itself.

Hm..., there is one in the TinyMCE, however. Looks we are lucky - we might merely add one to the widget - it is a matter of one line of code, as the widget is flexible enough and configurable. But we still won't be able to hook the "save" event, unless we provide a callback Java Script function. But our idea with the widget was - none of the JSNI code for widget users. So, we will hide it all inside. Here comes its usage concept:

TextEditor editor = new TextEditor();
// configure editor as needed (size, default content, etc.)
editor.addSaveHandler(new SaveHandler() {
  public void onSave(SaveEvent event) {
   ...
  }
});

For me - looks nice :) So, let's sum up what we need to create and manage a new custom event:

Create an event class - it may store any information related to the event, its source, attributes, etc.
Create an event handler interface - it will be implemented by widget users and it will be registered within the widget itself
Suite the widget with a method to register an event handler
And, sure enough, implement an event management in the widget

Add support for GWT events

Good news is that GWT already has almost everything what we need to manage events. GWT not only defines necessary interfaces and abstract classes, but even provides us with a HandlerManager - this "utility" will track all our event handlers and will fire events on them when we need it to. Thus, as we stick to GWT event model, we should implement our event and handler according to GWT standards.

Event

SaveEvent is an event fired up when user clicks save button. In fact it won't hold much information within itself, and so its implementation is pretty straightforward. We only need to subclass a GwtEvent, as we want to use GWT's event management means.

public class SaveEvent extends GwtEvent<SaveEventHandler>
{
  public static Type<SaveEventHandler> TYPE = new Type<SaveEventHandler>();

  @Override
  public Type<SaveEventHandler> getAssociatedType()
  {
   return TYPE;
  }

  @Override
  protected void dispatch(SaveEventHandler handler)
  {
   handler.onSave(this);
  }
}

Note that we added a TYPE constant and implemented getAssociatedType() method - this is needed as we will use GWT's HandlerManager and it will "map" events and handlers by their event type. We will also need this constant to be public - we will get to it in a moment.

Another one point to note is that event also implements a dispacth() method - otherwise how would GWT know the event handler's method to execute? Instead of trying to guess it - it delegates the event dispatching to us :) It is pretty simple anyway.

We used an event handler already... Here it is:

EventHandler

Similarly, as we use GWT's event management means, our SaveEventHandler will extend an EventHandler interface. Our interface itself is even simpler than SaveEvent - obviously it should be that simple as we want our users to start using it with minimal efforts:

public interface SaveEventHandler extends EventHandler
{
void onSave(SaveEvent event);
}

Register and track event handlers

Finally, we want to suite our widget with a way to add SaveEventHandlers to it. We agreed already that we will use a GWT's HandlerManager to manage "save" event. But we don't want our users to access it directly, rather we want them to use a convenient method, like addSaveHandler(SaveEventHandler).

public class TextEditor extends TextArea implements EntryPoint
{
  ...

  private HandlerManager handlerManager = new HandlerManager(null);

  ...

  public void addSaveHandler(SaveEventHandler handler)
  {
   handlerManager.addHandler(SaveEvent.TYPE, handler);
  }

  ...
}

Fire an event

Pretty cool, as you may notice we didn't much work to add an event. Well, it is not fired up anywhere yet, but not a huge work too. We'll see now.

In fact, TinyMCE already implements an ability to "catch" the user on clicking the "Save" button. So, this means we only need to add some JSNI code to do so, and link it together with our event model. To do so, we need to initialize the TinyMCE "save" plug-in, add a save button, and register so-called "save callback" function which is executed by TinyMCE. Hm, for me - I don't think it is a good idea to implement everything in the widget - in our implementation we will only ensure there is a "save" plug-in enabled, and add a callback function. We will let widget users to add a "save" button were they need it (and if they need it).

public class TextEditor extends TextArea implements EntryPoint
{
  ...

  private void initialize()
  {
   // initialize the stuff needed for "save" event
   addPlugin("save"); // ensure it is always there, even if widget user forgot to add it
   // (we also assume here at least "save" button is added - otherwise how would end user click it?)
   addOption("save_onsavecallback", "saveHandler");
   registerSaveHandler();

   ...
  }

  private native void registerSaveHandler()
   /*-{
   var thisTextEditor = this;
   $wnd.saveHandler = $entry(
   function() {
   thisTextEditor.@com.nimble.gwtaddons.texteditor.client.TextEditor::onSave()();
   }
   );
   }-*/;

  private void onSave()
  {
   handlerManager.fireEvent(new SaveEvent());
  }

  ...
}

That's it. Few comments about the callback we added - it may look a bit tricky, but it is not that much really. $entry produces a JavaScript function you may reference, and it may be executed on the instance all right, but the trick is that we should use an instance variable other than "this". It is because at the time the function is executed, "this" - will be a current window, but not a TextEditor instance. And thus we store it first to another one variable in the JSNI code, which assumes "this" to be a widget instance.

All right! You now have a working Save event! You may add a "save" button to a widget now, and add your SaveEventHandler. Try it out!

Resume

To sum up, adding a custom event is a pretty easy 4-step process: you create an event class, an event handler interface, reuse the HandlerManager to manage the events and handlers, and the only thing left for you is to implement firing up the event on a HandlerManager instance, and it will do the rest for you.

In particular, we added a SaveEvent to previously developed TextEditor GWT widget. You may download its source code and use it under the terms of LGPL 3.1 or higher.

Did you like the article?

Creating a full-fledged GWT Rich Text Editor widget

2010-06-25T17:48:00.339+03:00

Note: this post has a part 2 - Add custom events to your GWT widget or, "Rich TextEditor, part 2"

There is a plenty of Java Script libraries doing many useful stuff for you. And it is pretty easy to integrate them into your GWT project, so you obviously want to do so rather than reimplementing the same functionality in Java. But you still want to use as less JSNI code as you can, right?

In this article I would like to show how to create new GWT widget by wrapping existing Java Script library.

It was a need in a nice text editor widget when I created my first one. GWT comes with RixhTextArea which is useful to display rich text, but not that useful to edit a text when you realize that it doesn't come with the full set of controls to trigger text formatting (like buttons to trigger bolder text, drop-downs with fonts, etc.). So, you will need to implement all these by your own, if you stick to this widget.

On the other hand, there is a well-known TinyMCE JavaScript library (http://tinymce.moxiecode.com/) that does all the stuff for you. Well, in fact I think this is the reason why there is no pure GWT widgets with its functionality - why create one from scratch if we can reuse TinyMCE? So, that's what we will do here - we will create a TextEditor GWT widget reusing the functionality from TinyMCE. You can visit the page by link above and have a look on how it looks like, if you are not familiar with this library.

Widget interface

To get the idea on what it will be like, and what is a benefit of the separate widget, have a look at some sample code. The obvious benefit is no injection of any Java Script code to a host page. This is pure Java:

final TextEditor editor = new TextEditor();
editor.setWidth("700px");
editor.setHeight("400px");
editor.addOptions(TextEditorPreset.PRESET_ADVANCED);
editor.addPlugin("contextmenu");
editor.setText("Hello, <b>World</b>!");
editPanel.add(editor);

That's it! Nothing else. Once we have a widget implemented, we will be able to:

Add a text editor to a GWT application as a regular widget. No JSNI code at all, no changes to host HTML.
Use all the same functionality as for regular TextArea
Configure TextEditor with any options: select the controls to use, enable TinyMCE plugins (like spell checker, context menu, etc.).
Use "presets" to initialize the widget with a predefined set of options

And this is how the widget looks like:

Create a Project

Hereafter we will go through a widget implementation process, but I won't be much concentrating on the usual developments stuff, rather I will describe the general ideas behind the implementation. We also assume here that you have a GWT project for the widget. My project is named "TextEditor". To start with, our TextEditor.gwt.xml looks like:

I'd also recommend to create another one project where you will use the widget while implementing it. Create a panel with a widget, and launch this project in hosted mode from time to time to see what we have.

TextEditor widget base

Firstly, we need to identify the widget superclass. We have options to create a Composite, or a new widget from scratch, or extend an existing one. Obviously enough, we choose the latter: TinyMCE is a some kind of a "wrapper" over a <textarea> (well, not necesserily <textarea> only, though we will use it this way) - it needs to be initialized and linked to certain <textarea>s on a page. Well, it seems logical to implement our widget same way - it will be a TextArea subclass, and it will initialize TinyMCE over itself.

public class TextEditor extends TextArea implements EntryPoint
{
  // ...

  public void onModuleLoad()
  {
  }
}

As you notice, we also implement EntryPoint. In fact we do nothing in the onModuleLoad() implementation, but it's still necessary if we want to compile the widget.

Well, pretty good to time - our widget already has the standard interface and an implementation for such methods as setHeight(), setText(), etc. Pretty cool, but... oh, our widget does nothing new yet... Well, here goes...

Wrap the JavaScript library

Add the library to "public"

Let's add the text editor functionality now. To do so, download a TinyMCE library, extract it and add it to a new "public" package. Yes, that's the point - any necessary JavaScript libraries will reside in the project's public directory, and thus will be available with the widget itself. This means that you will not need to add the library to the \web directory if you are willing to use the widget. Sure enough, you should also add the new public directory to TextEditor.gwt.xml:

While the directory structure may look like:

  your\package\texteditor\TextEditor.gwt.xml
  your\package\texteditor\client
  your\package\texteditor\client\TextEditor.java
  your\package\texteditor\public
  your\package\texteditor\public\tiny_mce\
  your\package\texteditor\public\tiny_mce\tiny_mce.js
  etc. (the rest of the library)

Load the JavaScript to the host page

Well, as we added the library to the public directory for our project, sure enough we won't be willing to modify the host page to load it. Remember? Our idea is - not any manipulation on a host page. So, thus we load the library in the TextEditor.gwt.xml as well

Initialize the JavaScript library

O.K. Now we have a library included with the widget itself, it will be loaded to the host page, but it still has to be initialized. And so, we override the onLoad() method of the widget. It will launch a JSNI method in turn:

public class TextEditor extends TextArea implements EntryPoint
{
  private static final String DEFAULT_ELEMENT_ID = "textEditor";

  private boolean initialized = false;
  private String elementId;

public TextEditor()

  {
   this(DEFAULT_ELEMENT_ID);
  }

  public TextEditor(String elementId)
  {
   this.elementId = elementId;
   getElement().setId(elementId);
  }

  protected void onLoad()
  {
   super.onLoad();
   initialize();
  }

  private void initialize()
  {
   initTinyMce();
   initialized = true;
  }

  private native void initTinyMce()
  /*-{
   var options = {
   mode: "exact",
   elements: this.@your.package.texteditor.client.TextEditor::elementId,
   theme: "simple"
   }
   $wnd.tinyMCE.init(options);
  }-*/;
}

As you can see, we tell TinyMCE to work with "exact" textarea we specify, which is the one created by the widget itself. To be able to tell which one it is, we set the id to the textarea element in the constructor, and specify it to TinyMCE in the initTinyMce method.

You maybe noticed the initialized boolean variable - don't bother much now, we will see what it is for in a moment.

If you have a project to test the widget - try launching it now - do you like the idea so far? :)

But, oops... If you try using getText() or setText() now - you will realize it is not working. This is because of some TinyMCE specifics I won't concentrate on here. Let's add this:

  public String getText()
  {
   String result;
   if (initialized) {
   result = getContent(elementId);
   } else {
   result = super.getText();
   }
   return result;
  }

  private native String getContent(String elementId)
  /*-{
   return $wnd.tinyMCE.get(elementId).getContent();
  }-*/;

  public void setText(String text)
  {
   if (initialized) {
   setContent(elementId, text);
   } else {
   super.setText(text);
   }
  }

  private native String setContent(String elementId, String text)
  /*-{
     $wnd.tinyMCE.get(elementId).setContent(text);
  }-*/;

Now you see why we added an "initialized" boolean variable also. This is just to prevent us from trying to get content via tinyMCE API while the tinyMCE is not yet initialized. For example, it will be omitted when you invoke the setText(String) method before the widget is loaded onto a page.

Congratulations! You now have a working widget! If you try to launch sample application using this widget, you will be able to edit text and get the text in you sample application. Cool!

Polishing up - configure initialization options

Well, so far looks good, but still, our widget lacks any ability to configure it. You can not add controls, TinyMCE plugins, etc. So, lets add this ability. Again, let's see how TinyMCE works - we define the set of initialization options, invoke tinyMCE.init() passing it in, and so it initializes as specified.OK, so this means we need a way to manage these options. Previously they were hard-coded in the initTinyMce method, now they'll be configurable and will be built at the tine the initialization is invoked. Let's add the following:

  private Set fixedOptions = new HashSet(2); // options that can not be overwritten
  private JSONObject options = new JSONObject(); // all other TinyMCE options

  public TextEditor(String elementId)
  {
   this.elementId = elementId;
   getElement().setId(elementId);

   // fixed attributes
   addOption("mode", "exact");
   addOption("elements", elementId);
   fixedOptions.addAll(options.keySet());

   // load preset
   addOptions(TextEditorPreset.PRESET_BASIC);
  }

  public void addOption(String key, String value)
  {
   // do not allow overriding fixed options
   if (fixedOptions.contains(key)) {
   return;
   }

   options.put(key, new JSONString(value));
  }

  private void initialize()
  {
   initTinyMce(options.getJavaScriptObject());
   initialized = true;
  }

  private native void initTinyMce(JavaScriptObject options)
  /*-{
   $wnd.tinyMCE.init(options);
  }-*/;

Even though may look logical, we don't store options in a Map. The better for us is to store options in a JSONObject because later it will much easier to convert them to a options array accepted by tinyMCE.init() method. We also add several convenient methods to add new options. But we still don't want widget users overriding some of them - in particular these are the options related to how the TinyMCE finds the textares to link to. For this purpose we add a so-called fixedOptions set and check that users don't override them.

Pretty cool, yeah? Notice that we added pretty rich functionality with minimal JSNI tricks now.

That's it. Oh, almost forgot - do inherit from the com.google.gwt.json.JSON in your TextEditor.gwt.xml, as we use the JSONObject.

Polishing up - add configuration presets

Great, we may configure our widget. But, the configuration process may be pretty tricky if you need to add about a dozen of options. Well, you noticed we have a convenient method to add a bunch of options from a Map - addOptions(). So, you may easily store your favorite presets in a static Map and reuse it whenever needed. Hmm.. Why not adding some of the preset to the widget itself? Let's do - we create a new class:

public class TextEditorPreset
{
public static Map PRESET_ADVANCED = new HashMap();
public static Map PRESET_BASIC = new HashMap();

  static {
   PRESET_BASIC.put("theme", "simple");

   PRESET_ADVANCED.put("theme", "advanced");
   PRESET_ADVANCED.put("plugins", "emotions,spellchecker,advhr,insertdatetime,preview");
   PRESET_ADVANCED.put("theme_advanced_buttons1", "newdocument, |,
   bold, italic, underline, |, justifyleft, justifycenter,
   justifyright, fontselect, fontsizeselect, formatselect");
   PRESET_ADVANCED.put("theme_advanced_buttons2", "cut, copy, paste, |,
   bullist, numlist, |, outdent, indent, |, undo, redo, |,
   link, unlink, anchor, image, |, code, preview, |, forecolor, backcolor");
   PRESET_ADVANCED.put("theme_advanced_buttons3", "insertdate, inserttime,
   |, spellchecker, advhr, ,removeformat, |,
   sub, sup, |, charmap, emotions");
   PRESET_ADVANCED.put("theme_advanced_toolbar_location", "top");

   PRESET_ADVANCED.put("theme_advanced_toolbar_align", "left");
   PRESET_ADVANCED.put("theme_advanced_statusbar_location", "bottom");
  }

  private TextEditorPreset() {}
}

This is just an example. You may add any other options you like. You can find a detailed description of all supported options on the TinyMCE web site.

Finally, let's suite our widget with another one method to load a perset in. In fact, it is the same as addOptions(), but just looks more natural:

public void applyPreset(Map preset)
{
addOptions(preset);
}

Resume

That's it for this article. There is still lot to do with the widget, however. There are many ways to enhance it. Each next little step makes your widget even more flexible.

Well, to inspire you on this - I'll tell you a secret - as a proof of concept, this article was entirely written in the primitive WYSIWYG HTML editor created with the very same widget described here ;) True. Well, with some enhancements actually. If you are interested - in next articles I'll show the complete widget source code, how to manage custom plugins, how to manage save events, and many more. Just let me know ;)

Did you like the article?

Development Tests or "Don't wait for Test Team!"

2010-03-19T19:28:00.013+02:00

The issue I faced recently is that sometimes developers way too much rely on Test Team. I was there as well and I know how it works: you do very zen tests, commit your code, Test Team does necessary tests and... well, rejects it. I know that there are some developers who think this is a regular development process. Here I will describe what is wrong with this approach and why I think development tests are that important.

When you don't test your code...
Let's assume you implement a small feature. Once developed you run it one or two times to see if it works. Then you commit you changes.
Here is what happens next:

Test Team receives a notification and spends an hour or two to test the feature, find bugs, check the logs, etc. and finally submit defects to bug tracker.
You then receive a bug and spend some time to analyze it, recall how it works, etc. and spend an hour or two in total on a fix and commit changes again.

What happens next is that Test Team should do the same test again, so they spend 2 hours on that... And what if there are bugs again?

The point is that sometimes Test Team could find bugs only one by one. Let's assume you did 5 fixes. Each "iteration" took about 3 or 4 hours, then it will be 15 or 20 hours in total. But this process could spread on several days... To sum up, it seems that fixes are small, not more than hour or two, but as a result it takes several days to debug and finalize the feature! Wow!

What if you don't have Test Team? Then it means that your customers are your testers. Well, their time may cost even more. Not to say that some time your customer won't have a business with you if you deliver functionality that way.

To test or not to test?
At contrary, you could spend more time on tests. You won't be blocked by any defect because you could fix it at once.Yes, additional tests will take some time, but not more then bug fixing when bugs are raised by Test Team - because you then need to switch between your tasks.
Also this way you exclude the ping-pong in bug tracker and e-mail and free up much of Test Team time. Not to say that you improve your status with colleagues and your customer.

Conclusion
So, if you want to implement high-quality and reliable application - just bear in mind that Test Team is not to test your job, but to validate it. In ideal world, they should only approve that everything is all right and just close that ticket ;) Thinking that way helps me, hope it might be useful for you too

Did you like the article?

Fix bugs effectively in collaboration with the Test Team

2010-01-28T21:52:00.007+02:00

In this article I would like to share some of my personal hints concerning the case when customer reports a bug and this bug is not that obvious like when you have an error log. There are plenty of customer cases when there is only a functional issue, when something merely "goes wrong".

Let's suppose you are a developer who is assigned to fix this issue ASAP (well, you know, it is raised by customer - it is critical in most cases). What I want to tell is not "how to fix bugs", but how to collaborate with Test Team so as to do it effectively. In some projects it is mandatory to collaborate with Test Team in this kind of situations, but still some developers don't do that - maybe because they think the issue is not that complex, or by some other reason - it doesn't matter. I would like to point out why is it that important.

So, here goes... The very first thing you should do is

Notify Test Team about the issue

This step is really easy. You don't even need to do any analysis before notifying Test Team. But the benefit of that is really huge. In some cases your colleagues from Test Team will solve the entire problem for you.

When you notify test team you gain at least one more person who is doing a half of your job - he or she is trying to reproduce the error. Once the error is reproduced - you have a "full house" - it is only a matter of doing the same in your local development environment, then debugging and making a fix itself. In any case, you will try to reproduce the error on your side, won't you? So, while you do some preliminary analysis, there is at least one more person who already attempts to reproduce it.

Moreover, I had many cases when test team merely told me what the problem is, because they faced it before but didn't consider it as a defect. Hey, they indeed use the system more often rather than you do.

One more hint here is: let the person who is helping you with tests know your preliminary analysis results. Don't leave him or her alone - if you have more info about the issue - share it. It won't take much time, but will probably help your colleague.

Finally, even if they won't reproduce the bug, at least they will be prepared to validate your fix. I mean, you won't send a fix to a customer without having at least one more person who could tell that it works, will you? This leads us to next hint:

Let Test Team validate your fix

Well, this is a mandatory step. Once you succeed in analysis, debug and fix - you should validate it before claiming that it works. Never send a fix unless at least one more person reproduces the error without the fix, and then validates the fix by doing the same with the fix applied.

Ha, you can do that by yourself you say? Yes, indeed you can. In fact, you should - you should test your fix. But you should not be the one who does the final test - there is always a chance that your first fix won't work well. Let others validate it. Do you remember we've notified Test Team about the issue? Yes, even if they didn't reproduce the error for you, they already have most of the information they need to validate your fix (hm, or reject it :) )

We don't have Test Team - won't this work for us?

Indeed it is not that rare. In this situation - ask a colleague to play this role for a while. Note, that I don't mean asking a colleague to help you with the fix. It's not the same. Your colleague should work for a while as a Test Team member. It's not Pair Programming. The idea is that one person is dedicated to tests only. Well, if you usually work in pairs - all right, you can do that, but still ask one more person to help you with tests. After all, you want to have a quick and stable fix, don't you?

Conclusion

As I already mentioned, these steps are mandatory in some teams. But even then some developers don't do that. Sometimes they don't ask Test Team at all, sometimes developers only ask them to validate the fix, i.e. on last stage. It should not be so. These steps are fairly simple and won't take much of your time. If you spend about several hours on a fix, it is not a big deal to spend several minutes to share the information with Test Team or a colleague. But this raises your chances in finding a bug quickly and always increases the fix stability. If you do that already - you know that for sure. If you do not - give it a try - you will see the benefits yourself.

Did you like the article?

What can I do about incorrect specifications?

2010-01-26T13:53:00.012+02:00

If you work in the project where you receive requirements as formal documents like specifications, and the project is young enough – most likely that this is an article for you. It was a case in the project I work in, that sometimes specifications contain errors or the requirements are not detailed enough. This article would not discuss analysts and their job (which is really hard indeed) – let’s suppose that you already have received a document from them, and now have to implement something according to it. This article is the answer for question “what can I do about incorrect specifications as a developer?” There will be no any brand new exciting techniques in this article, but just a little of my experience, which helps me much in these situations.

Possible problems with specifications
I faced following problems with specifications
• the specification is not detailed enough, and thus it is not clear what exactly is supposed to be implemented
• there is a logic error in the requirements – customer requests something what is not logical at all and probably not the thing they need. Well, according to you :)

These issues could have many reasons. It is often the case when customer has a general vision, but analysts merely missed some details. In fact, it could be anything: it could be that customer’s demands depend on other parties, and the requirements are not clear on that “third” side; it could be a problem of reverse engineering; etc.; finally, it could be merely an error in the document.

“What can I do about it?”

Face it
Firstly, you should understand that this is absolutely all right that documents may contain some issues. Yes, it’s not good. But you should understand that analysts’ job is not that easy, and it is possible that they also can miss something. In fact, you do miss bugs in your application, aren’t you? So, give a chance for others to do that too :)

Ask
Next – just ask. Never leave it as is. You can use e-mail or chat or phone or whatever – but always contact the analyst and discuss the issue. And…

Don’t miss even “tiny” issues.
Yes, even if this issue is not that critical, or even if you think that you know what was supposed to be written there – still ask. Because you could be wrong too. It is only the customer who can make a decision about certain requirement.
Also, if you suppose something, it is possible that your colleague does not, though he needs to support you code with time. It is even possible that with time you can forget why did you decide to do the thing this way but not as described in the document. This is the case where you should be pedantic.

Propose solutions
Don’t just ask, instead - propose solutions. It is often the case when specification author doesn’t know possible standard solutions you used before even in the very same project. Or author doesn’t know about some standard details you usually need during development. In this case – share you knowledge with customer. Help analyst to create a correct document. Well, imaging for a moment you are an analyst. It’s really interesting!

E-mail conclusions
If you discussed in chat or by phone something that has an impact on project documentation – e-mail the conclusions to all discussion participants. I’m not sure if this is a case for you – but I never delete old messages in my inbox. This helps me much. Your mailbox becomes a second source of documentation.
Plus, all discussion participants will also have a chance to review conclusions and see if everyone was understood correctly.

Force specification author to update the document
This is mandatory. I mean it. Having documentation that is out of sync with your implementation is worse than no documentation at all. Other developers working on the same project might know nothing about your discussions and decisions you made with customer or analyst. They might even rollback your changes so as to align them to specification :) It is not acceptable. Finally, you yourself can forget about decisions you made.

Conclusion
In general, you should understand that analysts’ job is not that easy. Or, if you project has no analyst and customers provide documentation themselves – it is even harder for them. Help your colleagues in this hard work.

Did you like the article?

Imports in Python - possible pitfalls and how to avoid them

2009-12-19T13:40:00.002+02:00

Nonetheless Python is a very powerful language and supports all the stuff an enterprise-level language should support, to my mind it has some drawbacks coming from its nature - Python is an interpreted language. In this article I would like to highlight several pitfalls of python "import" statement and some hints and good practices helping to avoid them.

Recursive imports

Not a big secret how "import" works in Python (http://docs.python.org/reference/simple_stmts.html#the-import-statement). In general, when Python interpreter founds an "import" statement it loads and initializes the referenced module immediately (if not yet initialized). So, here is the consequence of that:

If a module is being initialized and interpreter finds another "import" statement in it, interpreter leaves this module partially initialized for a moment. Python starts to initialize another module (referenced by import) and as a result the module being initialized before is:

already marked as loaded (which means you can reference it and Python won't attempt to load it again)
but lacks the staff declared in it after other import statements

Well, this is what is going on "inside" the interpreter. But here is the consequence of that for our code:

Recursive imports (when A imports B and B imports A) do not work unless you let your module fully initialize before you import other dependent modules. That is a tricky description - better have a look on the example:


# file: commands.py
import helpcommand
...

class Command(object):
  def __init__(self, name):
    self.name = name

  def run(self, *args, **kwargs):
    raise NotImplementedError()

registry = (
  helpcommand.HelpCommand,
  ...
)

# file: helpcommand.py
import commands

class HelpCommand(commands.Command):
  ...

As a matter of fact the code looks natural, but obviously it is not working (well, it depends on it's usages - later about that). If you attempt to load commands module first, then Python will import helpcommand for you and will fail to define class HelpCommand because (ha!) there is no class commands.Command - its definition was not yet processed and interpreter knows nothing about it. As a result you get a message stating that module commands doesn't have a class Command. That's true, but not that obvious indeed.

There is a plenty of solutions for this case:

1. You can import dependent module first (helpcommand) - yes, in this case it works fine. But, whoa! - imports depend on the order they are being executed. I don't appreciate this solution because it is easy to brake it again - once you have to add "import commands" somewhere you first need to check if there was an import of dependent modules already. In fact, this means that you need to keep in mind the sequence in which your application modules are being initialized.

2. Another solution is to merely import all dependent modules only after their "root" module is initialized completely. For example, see the code below.


# file: commands.py
class Command(object):
  def __init__(self, name):
    self.name = name

  def run(self, *args, **kwargs):
    raise NotImplementedError()


import helpcommand
...

registry = (
  helpcommand.HelpCommand,
  ...
)

However, some dislike this solution because you put import statements somewhere in the middle of your source file. As for me - I'm OK with that, but anyway there is even better solution:

3. The best way to not fall into recursive dependencies with imports is to not use them. Indeed - in most cases you don't really need to do that, so in fact this "import" behavior is not a problem at all if you use it correctly. For example, in our particular case, let's split the logic in commands module to 2 separate modules, and see what happens:


# file: commandsregistry.py
import helpcommand
...

registry = [
  helpcommand.HelpCommand,
  ...
]

...


# file: commands.py
class Command(object):
  ...


# file: helpcommand.py
import commands

class HelpCommand(commands.Command):
...

Now, whenever you import any module, it works fine, because you don't have any recursive dependencies between the modules in you application. The solution is - do not mess up the stuff.

Shared recources initialization

Another one potential problem with imports in Python is that some resource could be initialized incorrectly before you want it to. For example, consider this:


# file: launcher.py
import logging
import logging.config

import mymodule

if __name__ == "__main__":
  logging.config.fromFile("logging.ini")
  mymodule.doStuff()


# file: mymodule.py
import logging

logger = logging.getLogger("mymodule")
...

Once you launch your application with launcher.py, it is first loading all modules it depends on (in the case above) - thus loads mymodule before the logging was initialized correctly. Thus, logger initialized in mymodule is initialized with default logging configuration, and will not be able to log anything even after the launcher had initialize the logging system. Again, this means that you need to keep an eye on the process of the system initialization - the sequence of imports and shared resources initialization. Eh, but we don't want to do that, so you can solve this in several ways:

1. Lazy loading. For example, instead of initializing the logger in mymodule once it is loaded, do a lazy loading:


_logger = None
def logger():
  if not _logger: _logger = logging.getLogger("mymodule")
  return _logger

2. Again, you can import dependent modules only at a time you really need them. Just put "import mymodule" after your base system initialization. But, again, we can go further. In general we could clearly split the initialization logic and your system logic. For example, in this case that would be:


# file: main.py
if __name__ == "__main__":
  try:
    import initialization
    initialzation.init()
  except Exception, ex:
    ...
  else:
    import launcher
    launcher.main()


# file: launcher.py
import mymodule
import ...

def main():
  mymodule.doStuff()
  ...


# file: initialization.py
import logging
import logging.config

def init():
  logging.config.fromFile("logging.ini")


# file: mymodule.py
import logging

logger = logging.getLogger("mymodule")
...

Resume

Python's import is not that simple. Yes, it merely loads the module you ask him to. But, it is easy to find yourself solving bugs caused by its incorrect usage and you will need to constantly keep an eye on the sequence in which you import modules in your system... That is not what you are supposed to spend your time for. From the very beginning write your code without messy and cumbersome dependencies between parts of your system, follow few good practices and you will most likely avoid many potential problems described in this article.