Dev-Picayune

Okay, very weird problem today. I am not sure what to make of it… when using the latest version of MySQLdb (1.2.2.final.0) which is the version in Ubuntu 7.10 Gutsy, I am getting an “incomplete format” error when performing an executemany. My variables are presented as a list of dictionaries as I’m using named/mapped variables in the SQL statement. Using the exact same code with MySQLdb (1.2.1.final.2) is fine (the version in Ubuntu Feisty 7.04).

So for completeness sake, I have documented the scenario like this:

On a MySQL server, you just need a simple little table:

CREATE TABLE `test_table` (
`id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`test_col_1` varchar(20) NOT NULL,
`test_col_2` varchar(20) NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=latin1

Then the Python code (obviously you’ll need to set the connection stuff appropriately):

import MySQLdb
 
print "-----------------------------------------------------"
print MySQLdb.version_info
print "-----------------------------------------------------"
 
sqlStringItems = """INSERT INTO test_table
(test_col_1,test_col_2)
VALUES
(%(test1)s, %(test2)s);"""
 
dtlDict = {'test1':'test value 1','test2':'test value 2'}
 
con = MySQLdb.connect(host="localhost",user="myuser",passwd="mypassword",db="test")
cur = con.cursor()
exitval = cur.executemany(sqlStringItems,[dtlDict])
 
print "exit value from executemany= " + str(exitval)
cur.close()

version (1, 2, 2, ‘final’, 0) yields:

ValueError: incomplete format

Whereas version (1, 2, 1, ‘final’, 2) works fine.

Also, doing the same thing, but with execute instead of executemany works fine (and of course taking the dict out of the list). So this seems to be purely an issue with executemany. Looking at the MySQLdb code, it’s obvious that between the two versions it was pretty radically changed.

I guess I need to report this as a bug, but I’m just finding it hard to believe that I didn’t do something wrong.

Update (2008-06-03)
Back in January, I reported this issue (1874176). Then in late April, Raphael Guillet submitted a possible fix. I haven’t spent much time looking at the right vs. wrong of the fix, but I have tried it and it appears to work. So for those looking, here’s the possible fix which as near as I can tell has still not been cut into the official version.

The change is in the cursors.py file. So you have to hunt that down wherever it is stored in your particular OS version and/or install.

Comment out lines 201 and 202:

        #e = m.end(1)
        #qv = m.group(1)

And then add right after that:

        e=len(query)
        qv = query[p:e]

I’ll try to continue to monitor this.

(now updated with some pointless Python code)

This is perfect for this little blog concerned with trivial and small things. I noticed that my Peanuts calendar on my desk is essentially a reproduction of the 1990 calendar. Obviously, they all have to be reproductions because Charles Schulz died in 2000. So looking more closely, I realized they’d chosen 1990 primarily because the dates and days of 1990 match 2007. So a quick hack with the ‘cal’ program under linux and I was off and searching for what will most likely be next year’s calendar. Since 2008 is a leap year and obviously 1991 is not, I knew that wasn’t going to work.

First I generated target calendar with:

cal 2008 > cal2008.txt

Then, I just manually (although I could have written a script) looped through a few known leap years and found that 1980 seems to match perfectly as well.

cal 1980|diff - cal2008.txt

Obviously, what we’re looking for is a diff of only the first line, the year. Side note: the selection of 1952 (also a match), would seem to be fairly unlikely.

For 2009, there are a bunch of choices including: 1953, 1959, 1970, 1981, 1987 and 1998.

Why am I posted this? I don’t know other than it was fun to use ‘cal’ and ‘diff’ to anticipate which Peanuts calendar would be next.

UPDATE

I couldn’t resist doing a Python solution to this little problem. I know this isn’t as efficient as someone else could make it, but it was entertaining for me to do it. It took about 5 or 6 minutes to write and ran correctly the first time (matching my previous results):

import sys 
import datetime
def matchCals(inputYear,minYear,maxYear):
    sDayOfWeek = datetime.date(inputYear,1,1).weekday()
    eDayOfWeek = datetime.date(inputYear,12,31).weekday()
    for testYear in range(minYear,maxYear+1):
        if (sDayOfWeek == datetime.date(testYear,1,1).weekday() and eDayOfWeek == datetime.date(testYear,12,31).weekday()):
            print "Match Found... %s = %s" % (str(testYear) , str(inputYear))
 
if __name__ == "__main__":
    matchCals(int(sys.argv[1]), int(sys.argv[2]), int(sys.argv[3]))

Here I just check the day of the week on the first day of the year, and the last day of the year. If both of those fall on the same day, I assume the calendars for those years are the same. And since we’re dealing with a limited set of dates (valid complete Peanuts calendars running roughly 1951 to 1999, it handles a min and max year for input as well. So to run it for 2008, you just type in:

python ./caltest.py 2008 1951 1999

Of course, there is no error checking… so if you enter a string or an invalid year… that’s your own problem. I just wanted to see how quick I could do it. Obviously, I’ve waisted more of my time… and more of your time.