Using Python for existing Python code migration

Home/Python/Using Python for existing Python code migration

Keeping software design tidy is not an easy task, especially when the code base grows significantly. In some cases systems are composed of thousands of modules, which in specific moment need to be updated, migrated or optimized. It is possible that a huge part of the code is auto-generated Python code and it would be a huge hustle to migrate all that code manually. In any case the work needs to be done and it clearly needs to be automated.

One option would be to use Regular Expressions to find existing patterns. You start describing different migration cases and hooking them to callback functions that need to get the job done. At the end you end up with a huge amount of scenarios that are extremely hard to maintain. The replace needs to happen in place and you risk that the already migrated code will be fetched by the next Regular Expression. Then you start again.

Fortunately, Python offers a better way to do it, hidden in the standard library and called lib2to3. Yes, this is a tool for migration from Python 2 to 3, it could be adapted to our needs. For the purpose we just need to extend it with our own fixers. Fixers allow you to directly modify the Parse Tree be changing values of Nodes and Leafs, deleting them and inserting new ones.

In order setup your code migration project you need to copy lib2to3 from the standard library in your workspace, delete all fixers from the lib2to3/fixes folder (unless you also want to migrate to Python 3) and you are ready to start writing your own migration rules. Writing your own fixers this way will allow you to keep migrations organized and encapsulated, making them more maintainable.

Here is the following snippet, just to get an idea how a fixer looks like:

from lib2to3.fixer_base import BaseFix
from lib2to3.pgen2 import token

class FixName1(BaseFix):
    
    _accept_type = token.NAME

    def match(self, node):
        if node.value == 'oldname':
            return True
        return False
    
    def transform(self, node, results):
        node.value = 'newname'
        node.changed()

Written by

The author didnt add any Information to his profile yet