Lazy getters

[pdonadeo]

Another important point, now missing in this project, is the availability of some sort of "lazy getter". I have in mind something like the Enum module present in Batteries. The x_get (or x_get_enum) function should return an enumeration, so that a "SELECT *" on a large dataset wouldn't blow up the server.

This is a naive (but working) implementation for the Sqlite3 backend:

open BatStd;;

module MySqlite3 =
  struct
    include Sqlite3

    let enum (db : Sqlite3.db) (query : string) : Sqlite3.Data.t BatMap.StringMap.t BatEnum.t =
      (* stmt creation *)
      let stmt = Sqlite3.prepare db query in

      let rec prepare_tail stmt =
        match Sqlite3.prepare_tail stmt with
        | None -> ()
        | Some s -> prepare_tail s in
      prepare_tail stmt;

      (* Enum "next" *)
      let next () =
        try
          match Sqlite3.step stmt with
          | Sqlite3.Rc.ROW -> begin
              let col_names = Sqlite3.row_names stmt in
              let col_data = Sqlite3.row_data stmt in
              let len = Array.length col_names in
              let map = ref BatMap.StringMap.empty in
              for i = 0 to (len - 1) do
                map := BatMap.StringMap.add col_names.(i) col_data.(i) !map;
              done;
              !map
            end
          | Sqlite3.Rc.DONE -> begin
              Sqlite3.finalize stmt |> ignore;
              raise BatEnum.No_more_elements
            end
          | e -> begin
              failwith (Sqlite3.Rc.to_string e)
            end;
        with
        | Sqlite3.Error "Sqlite3.step called with finalized stmt" ->
            raise BatEnum.No_more_elements
      in (* let next ... *)
      BatEnum.from next
  end
;;

It returns an enumeration of maps representing records, with the keys (strings) populated with the columns name, and the values being the corresponding Sqlite3 values. You could return the "X" type, of course.

[samoht]

So I've pushed to my repository something which generate 't_lazy_get' which takes the same arguments as t_get, but which returns a 'unit -> t option' function instead of a 't list'. You can call the function until it returns None which means no more elements are in the query. Hopefully that should do what you want.