Dev-Picayune

I finally made a build of my game, Raven Checkers, on Windows XP. In the process I ran into the same issues as everyone else with Python 2.6 and the new requirement for manifest files. Since Python 2.6 binaries for Windows are now compiled using Visual Studio 2008, a manifest file is required both for your own executable and for the C runtime that Python uses. I was not able to find a simple step-by-step process on how to do this, so here’s how I do it.

1. Make sure that Python 2.6 is installed on your system with the “Install just for me” option during setup. This ensures that the DLLs and manifest files that you will need will be placed into the Python26 directory where you can easily locate them later.
2. Build an executable of your program for Windows. I’m using gui2exe (0.3) and py2exe (0.6.9) to make it simple. I set the Bundle Files option to 2, and the Compression option to 0. Why, do you ask? I am unable to get any other combination of options to work properly. If you can, more power to you. I also use the option to create a dist directory called, um, ‘dist’.
3. From the Python26 directory, copy msvcr90.dll, Microsoft.VC90.CRT.manifest, and python.exe.manifest over to your dist folder.
4. Rename the python.exe.manifest file to [yourappname].exe.manifest.

Now you should be able to copy your dist folder over to a clean Windows machine, double-click on your executable and have it start without any side-by-side configuration issues.

By the way, If you don’t have a clean Windows machine to test on your app on, the easy and free solution is to download Microsoft Virtual PC 2007 and a free XP or Vista virtual machine image. Once VPC is installed, you can copy your files to your virtual machine by simply dragging them from your dev machine desktop onto the VPC desktop window.

Here’s to Windows deployment and testing made (relatively) simple! Hope this saves you some hours of Googling.

I’ve never really regretted keeping two separate blogs. Three was certainly too much, but having two isn’t all bad. The one thing I’ve regretted is that I’ve not done enough posts. So, to help fix that situation and also to provide a slightly more permanent home to the excellent posts that he writes, I’m pleased to announce that Brandon Corfman is now going to be co-author, if not primary author, here on this site. Brandon brings both a more academically grounded education background together with even broader practical experience programming.

Brandon is single-handed responsible for my initial interest in Python as well being the greatest of friends. So now it’s official, this site, now has gained a whole lot more credibility. At the same time, perhaps Brandon’s input will spur me on to write a few more things here as well. Enjoy!

This issue has been documented before, but it took me awhile to understand what the real problem was and what my options for resolving it were.

So what happened? I had a VM (Ubuntu Hardy) chillin’ on my local dev box. That local VM was a test environment to verify the performance of a django app being hosted on a VM with Apache and mod_wsgi. And wouldn’t you know it? It was slow… very slow. As in 5 seconds per browse to display or anytime it had to hit the database. Dropping to the command-line revealed that it was the actual connecting to the mysql server (which is a separate server) that was taking most of all that time.

As it turns out, MySQL’s name lookup (reverse address resolution) was causing the slowdown. You can turn off MySQL’s reverse lookup using the “–skip-name-resolve” option. But since changing the configuration of a running MySQL server (production) is not an option, I had to figure out something else. The VM was getting a DHCP address and since our in-house (Windows-based) DNS doesn’t do well at tracking the names of DHCP’d clients that are Linux boxes, our DNS stinks at getting the name in any timely fashion. But, if you assign the client a static IP that has an official name in our DNS, voila! You have a nice, fast connection time again. A Windows box in my office, that gets assigned an IP via DHCP is fine as well, presumably because our DNS server plays well with other Windows boxes.

As I stated earlier, you can disable the name lookup but then you have to use IP addresses if you want to support security based on the connecting client’s machine. Making sure there are DNS entries for the clients seems to fix it. Again, this is mostly just for my own record if nothing else. I certainly don’t want to see this crop up again and not remember what I figured out.

So after hearing about CherryPy’s WSGI server while at PyCon (I went to the Pylons/TG2 classes), I decided, like others have, to see if I could host a Django site with it here where I work. There are several references out there about using that server code with Django, but I had a tough time getting to the point where I could actually use it to handle the Admin media. Some of the articles were slightly out of date with changes made to the most recent version of the CherryPy WSGI server, so while this is drawing off of several other folk’s work (like Eric Florenzano and DjangoCerise ), I present what I did to make it work while also realizing that this is probably still very incomplete. I am just hoping this will help someone get started.

Prior to the code, I downloaded just the wsgiserver.py from CherryPy. You can get the latest from trunk if you’re running Linux, with just a command like this:

wget http://svn.cherrypy.org/trunk/cherrypy/wsgiserver/__init__.py -O wsgiserver.py

Then I also wanted to have some logging capabilities, so I discovered Paste’s TransLogger middleware. All you have to do is download the translogger.py file from the paste project (sorry no quicky command for that, I grabbed it via the browser and I am too lazy to get you the command for that).

So with those 2 files (wsgiserver.py and translogger.py) now in the main directory of my particular django project (dash2 is the name in my example), I created a new cherryserve.py in that same directory as well.

import wsgiserver
#This can be from cherrypy import wsgiserver if you're not running it standalone.
import sys
import os
import django.core.handlers.wsgi
from django.core.servers.basehttp import AdminMediaHandler
from translogger import TransLogger
 
if __name__ == "__main__":
    sys.path.append('/home/swilcox/projects')
    os.environ['DJANGO_SETTINGS_MODULE'] = 'dash2.settings'
 
    app = AdminMediaHandler(django.core.handlers.wsgi.WSGIHandler())
    logged_app = TransLogger(app)
    server = wsgiserver.CherryPyWSGIServer(	
        ('127.0.0.1', 8080),
        logged_app,
        server_name='luz.lifeway.org',
        numthreads = 20,
    )
 
    try:
        server.start()
    except KeyboardInterrupt:
        server.stop()

So why did I do the path trickery up there? Well, because in my development environment, I don’t want my entire projects directory normally on my python path. Feel free to call me paranoid. But to get the server to work correctly with the django code, I needed it there. For a production server, the sys.path.append business wouldn’t normally need to be there, because I would have my apps or projects directory as part of the Python path.

Also note that if you needed to serve multiple apps, with this newer version of the wsgiserver.py, you would want to set them all up in a WSGIPathInfoDispatcher object first and then pass that in to the CherryPyWSGIServer instead of logged_app.

I think the other fields are pretty self-explanatory… ports, IP address, threads, etc…

The Admin Media trick is using app = AdminMediaHandler(django.core.handlers.wsgi.WSGIHandler()) instead of just the WSGIHandler by itself.

If you don’t want the logger, you just pass in app instead of logged_app.

So just as a bookmark to myself, I wanted to document what all I was working on from a technical perspective.

On the work front, I’ve recently deployed a Django app that folks are just totally loving. It’s actually a hacked together group of scripts to import data and export data to two different legacy systems. In the middle, I use the Django admin stuff to allow folks to maintain the data, and scripts at the beginning and end of the day handle import and export duties.

I finally started hacking around on WordPress themes. I’ve not gotten very far but hopefully in the next week or so, I’ll have new designs for both sites.

Then, with regard to PyCon, I submitted some talk proposals. Not sure I should have bothered, but it seemed like the thing to do. Now I’m just curious to see if any of my proposals are accepted. One of the proposals is for a project I’ve only begun really gathering the pieces for. I’ve resolved that whether the talk is approved or not, I’m going to work away at the project and then I’ll either have a talk or I’ll have a lightning talk to present. I’ll be blogging my way through my progress here, so should have some interesting stuff to post.

I’ve finally gotten tired of my poor little blog getting attacked by spammers. Actually, it was just one in particular and I know I could’ve done some IP blocking or added to the protection scheme on my comments, but rather than working more on Pumpkinvine, I’ve now switched to WordPress. And while it seems like I’m giving up, it’s really so I have the chance to catch my breath while I use all the WordPress plugins to handle the spam for me. But for reference, I ought to package up my most recent version of Pumpkinvine and make it available for folks although it’s really a pretty rough collection of PHP scripts that approximate the same style as blosxom. So why bother creating another system? But it seemed like a good idea at the time. Probably the neatest part of Pumpkinvine was going to be the switchable back-ends so you could choose between text files, SQLite, MySQL or something else by just changing a single setting. Of course, the only back-end I had done was the text files-based core.

Ultimately, I was tired of messing with PHP. I’d like to go with more Python. Django appears to the fastest way to get to having a working blog, but lack of personal free time, and lack of desire to fight with FastCGI and Dreamhost to get up and running, kind of dampened my spirits. So I’ll settle for my clever Python script that I wrote to convert from Pumpkinvine to WordPress and some time that I’ll need to devote to learning how to create WordPress themes.

I found myself needing to do a fairly complicated thing. Well, it’s not complicated, but for a DOS batch file, it seemed complicated. The problem I was facing: to insert some “official” DOS command stuff into a .bat file that is run on all computers in our entire organization. I have no idea how many machines were talking about here but I would guess the number to be in the 5,000+ range or higher. Specifically, this is our official domain login script. Yikes. I don’t like having anything to do with something that gets touched this often. One screwup in there and it’s not a good thing. The problem is that we have a special group of machines (about 130) that are not registered on the domain. Now we need them on the domain but we don’t want to run some of the programs that our “Software Taliban” uses for monitoring machines. All of these special machines have a common naming scheme which is the letter S followed by a 7-digit number. So my task was insert some DOS .bat stuff in the existing .bat file that will run without external dependencies (other than whatever is already guaranteed to be there). All I knew for sure was that the environment variable COMPUTERNAME would be filled with the name of the computer and that my special machines were running 2K or XP and that either way, the OS environment variable would be filled with “Windows_NT”.

I also wanted to make sure I didn’t catch machines that were named “Snoopy” or anything like that. So for lack of a better way, I chose the following as my DOS string substitution method:

SET CN=%COMPUTERNAME% 
SET CN=%CN:0=x% 
SET CN=%CN:1=x% 
SET CN=%CN:2=x% 
SET CN=%CN:3=x% 
SET CN=%CN:4=x% 
SET CN=%CN:5=x% 
SET CN=%CN:6=x% 
SET CN=%CN:7=x% 
SET CN=%CN:8=x% 
SET CN=%CN:9=x% 
IF /i "%CN%"=="Sxxxxxx" GOTO END

The magic here is the funky %variable:y=x% stuff. Essentially, it’s string replacement where any ‘y’ is replaced with an ‘x’. This then normalizes the name into something I can compare against. Doing it this way also guarantees that I am comparing the length as well. So while it isn’t pretty, I was pleasantly surprised to find it was possible to do in DOS. There is also a method for getting a substring using a similar syntax: %variable:~s,l% where s is the starting position (1-based) and l is the length of the sub-string. Talk about obscure. But then again, how often do you have to stick to plain old DOS .bat files?

Technorati Tags: DOS, .bat files

For my reference, and the reference of anyone who needs to find it, while playing around with the xmlrpclib module, I found myself needing to be able to adjust the timeout. As near as I can tell, if you don’t set a timeout value, it will run forever or at least a long time (I have never been willing to wait long enough to find out). The trick to giving xmlrpclib a timeout on calls to the server is importing and using the socket module. The following code example essentially covers what you can do:

import xmlrpclib 
import socket 
s = xmlrpclib.Server('http://myRemoteServerName/myEntryPoint') 
s.myProcedureName('hello input')    #this call will not timeout... 
socket.setdefaulttimeout(10)            #set the timeout to 10 seconds 
s.myProcedureName('hello input')   #now the call will timout after 10 seconds 
socket.setdefaulttimeout(None)       #set it back to no timeout value (if you want to)

The kicker here is just setting the defaulttimeout value in the socket module with the setdefaulttimeout method. If the RPC call times out, it will throw a timeout exception that you’ll need to be able to handle. But at least that’s better than the alternative of not ever timing out.

Technorati Tags: python, xmlrpc, xmlrpclib