Started using sqlite the way you're supposed to, with an INSERT OR IGNORE rather than checking with a SELECT statement to see if the row was already in the database. Holy shit, roughly x10000 faster. I should read the documentation more.