Today I made an important change to the python-apt code: It is now native Python 3 code (but also works under Python 2). The previous versions all run 2to3 during the build process to create a Python 3 version. This is no longer needed, as the code is now identical. As part of that change, python-apt now only supports Python 2.7, Python 3.3, and newer. I’m using some features only present in 3.3 like Python2 unicode literal syntax in order to keep the code simple. ... Read more 》

Writing recursive descent parsers is easy, especially in Python (just like everything is easy in Python). But Python defines a low limit on the number of recursive calls that can be made, and recursive descent parsers are prone to exceed this limit. We should thus write our parser without real recursion. Fortunately, Python offers us a way out: Coroutines, introduced in Python 2.5 as per PEP 342. Using coroutines and a simple trampoline function, we can convert every mutually recursive set of functions into a set of coroutines that require constant stack space. ... Read more 》

You can use a PBO solver to clean up your system from unneeded automatically installed packages. First of all, you convert the system state to PB, and add an optimization function telling it to remove as many automatically installed packages as possible. Then you run this thing through a solver (such as clasp, which seems the fastest solver for PBO instances in the Debian archive) and convert its output to human-readable package names. ... Read more 》

Today, I got an email from an user of one of my Python scripts asking why the script t does not work on Arch Linux anymore. As it turns out, the Arch Linux team decided to switch /usr/bin/python to Python 3.0 and use python2 for Python 2.X versions. By doing this, they decided to make their distribution incompatible to almost any Python script in the world. Arch Linux’s decision to diverge from the rest of the world that uses python for Python 2.X and python3 for Python 3.X is stupid. And doing this without updating reverse dependencies beforehand and thus breaking packages in their own distribution is insane. In the end this means that if you use Arch Linux, you should consider switching to a distribution that does things the right way: Debian. You can also switch to Ubuntu, that’s normally just a bit less right (= faster). But using a distribution that does those crazy things in such an irresponsible way is really insane. ... Read more 》

Last year, in July, I reported an issue to Python’s issue tracker. This issue can be seen at http://bugs.python.org/issue6483. Since then, there has been no action on this bug from the developers. The bug describes that every object stored in a module will not be deallocated if the module is deallocated and it’s m_size = -1 (which it should be if the module has no state). The problem seems to be that Python copies the dictionary of the module but forgets to decrease the reference count of the copy when the module is deallocated. This bug may have serious impact if objects are stored in the module which write status to a file when they are deallocated, because the deallocation functions are never called. ... Read more 》

So, I have started to port python-apt to Python 3. Most things work already, but there is one single problem. I can not access the attributes of the objects, only their methods. In Python 2.5, everything works perfectly. In Python 3.1, the same code produces an error. An example is apt_pkg.GetCache().Packages. I defined the slots tp_getattro and tp_methods. In Python 3.1, tp_getattro seems to be ignored. If you want to help, http://bzr.debian.org/loggerhead/users/jak/python-apt/py3k/changes for browsing the branch and http://bzr.debian.org/users/jak/python-apt/py3k/ for branching it. ... Read more 》

News: http://bzr.debian.org/loggerhead/users/jak/python-apt/jak/revision/219 introduced again some new documentation. doc/source/apt/debfile.rst | 6 doc/source/apt_pkg/cache.rst | 540 +++++++++++++++++++++++++++++++++- doc/source/apt_pkg/index.rst | 24 + doc/source/conf.py | 5 doc/source/examples/cache-packages.py | 22 + doc/source/examples/cache-pkgfile.py | 29 + doc/source/examples/missing-deps.py | 51 +++ 7 files changed, 659 insertions(+), 18 deletions(-) You can see them at http://people.debian.org/~jak/python-apt-doc/ Missing seem to be: AcquireFile AcquireItem ActionGroup Configuration MetaIndex PackageIndexFile PkgManager PkgRecords PkgSourceList PkgSrcRecords ProblemResolver TagFile TagSection But the weekend is not over yet. I will blog tomorrow again, and once I’m finished with everything (which may be tomorrow, too). ... Read more 》

I have been working the whole week on python-apt and the result is the jak branch. This implements some of the proposals I made in my last post, but has one very interesting feature: REAL COOL DOCUMENTATION. After Sandro Tosi told me in a comment in my last post that the real big problem with python-apt is a lack of documentation, I immediately started writing it. UsingreStructuredText and Sphinx, we now have a really cool and much more detailed documentation. (Although it is not really finished yet [it contains everything, but there is still room to improve]). ... Read more 》

Well, this is my second post about speed in Python. Today, I noticed that debimg’s dependency resolver was much much slower than before. I thought what the problem could be and finally realized that the problem was that I switched from sets to list. This is fixed now in commit d0fd700080de5c19cb5fd66918d14c5ffa26e805 Now, some benchmarks (using IPython): In [1]: a = range(10**6) In [2]: b = set(a) In [3]: %timeit 10**6 in a 10 loops, best of 3: 31.8 ms per loop ... Read more 》

A first preview of the python-libisofs bindings is available now. It’s currently located in a git branch at git.debian.org, but this may change at a later point. The bindings support the creation of ISO Images and (almost) all options libisofs supports, like Rockridge, Joliet, and much more. Reading and Modifying existing images is not supported yet. The code is written in Cython and you need cython installed for building from the git branch. It can be installed just like any other Python module/extension/package, using a setup.py. ... Read more 》