SQLite fetchrow

Matts on 2003-01-30T11:40:52

At the moment in DBD::SQLite I have to use the API to get an entire select results into memory and then loop through that for fetch() calls. This is a bit annoying because it uses lots of memory for large result sets.

Internally SQLite implements a callback mechanism (sort of like SAX) but this doesn't work with the DBI's model of working, so we have to use the above.

Thankfully the SQLite team has added a non callback streaming API into CVS, so at some point I'll convert DBD::SQLite to use this.

Even more interesting is that the model looks almost exactly like how the DBI is written, so the code will get even simpler in the next release. Woot.


maybe not an accident

merlyn on 2003-01-30T15:58:41

You wouldn't think maybe perhaps the SQLite developers hadn't noticed DBD::SQLite, and said "gee, we need to interface to that better?"

Re:maybe not an accident

jdavidb on 2003-01-30T18:03:02

I would expect either Matt told them what they needed to do, or else the SQLite developers naturally figured out on their own that there was a better way of doing things. If it works well for DBI, it'll work well for other software packages, too.

Re:maybe not an accident

Matts on 2003-01-30T19:53:43

I don't actually have that much contact with them. However I think developers of other packages do - such as the Delphi and ODBC developers. I think pretty much every high level database access layer works like DBI, so the model they chose only maps well to custom apps.

Good to see it developing so fast though. Maybe next we'll be able to store binary data (if only!).

Re:maybe not an accident

acme on 2003-01-31T10:39:04

Binary data is one of my wishes for SQLite. It's so retro that it only accepts c-strings. Maybe one day ;-)