From erik at hollensbe.org  Mon Sep  1 17:11:16 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Mon, 1 Sep 2008 14:11:16 -0700
Subject: [ruby-dbi-users] dbd_pg and deallocating prepared statements
In-Reply-To: <5d847bcd0808261856p5d77328chf53c14db2edbf3ad@mail.gmail.com>
References: <5d847bcd0808261856p5d77328chf53c14db2edbf3ad@mail.gmail.com>
Message-ID: <200809011411.16704.erik@hollensbe.org>

On Tuesday 26 August 2008 18:56:40 KUBO Takehiro wrote:
> Pardon me if I'm not correct. I have been interested in how various
> DBMSs implement prepared statements. But I have not used postgresql
> for 9 years.
>
> dbd_pg uses prepared statements for all queries. But who deallocate the
> prepared statements? "@stmt.clear if @stmt" doesn't do it.
> IMO, if a ruby process uses one database session for all SQL statements
> and issues them periodically, server side (postmaster) memory will be
> increased as time goes.
>
> See: http://www.postgresql.org/docs/8.3/static/libpq-exec.html#AEN30965
>      http://www.postgresql.org/docs/8.3/static/sql-deallocate.html

This bug (and a type handling bug in mysql) have been fixed in the recent git 
push. If anyone has some spare time and would just like to run the test suite 
which should reveal any issues in either of these changes, it'll ease my mind 
to release it today.

Thanks again for the help in solving this problem.

-Erik

From kubo at jiubao.org  Tue Sep  2 01:39:09 2008
From: kubo at jiubao.org (KUBO Takehiro)
Date: Tue, 2 Sep 2008 14:39:09 +0900
Subject: [ruby-dbi-users] dbd_pg and deallocating prepared statements
In-Reply-To: <200809011411.16704.erik@hollensbe.org>
References: <5d847bcd0808261856p5d77328chf53c14db2edbf3ad@mail.gmail.com>
	<200809011411.16704.erik@hollensbe.org>
Message-ID: <5d847bcd0809012239n48fcb4a7r3124773e34dd68f7@mail.gmail.com>

Hi,

On Tue, Sep 2, 2008 at 6:11 AM, Erik Hollensbe <erik at hollensbe.org> wrote:
> On Tuesday 26 August 2008 18:56:40 KUBO Takehiro wrote:
>> Pardon me if I'm not correct. I have been interested in how various
>> DBMSs implement prepared statements. But I have not used postgresql
>> for 9 years.
>>
>> dbd_pg uses prepared statements for all queries. But who deallocate the
>> prepared statements? "@stmt.clear if @stmt" doesn't do it.
>> IMO, if a ruby process uses one database session for all SQL statements
>> and issues them periodically, server side (postmaster) memory will be
>> increased as time goes.
>>
>> See: http://www.postgresql.org/docs/8.3/static/libpq-exec.html#AEN30965
>>      http://www.postgresql.org/docs/8.3/static/sql-deallocate.html
>
> This bug (and a type handling bug in mysql) have been fixed in the recent git
> push. If anyone has some spare time and would just like to run the test suite
> which should reveal any issues in either of these changes, it'll ease my mind
> to release it today.
>
> Thanks again for the help in solving this problem.

The following patch will release prepared statements in GC even though they are
not finished explicitly.
   sth = dbh.prepare('select * from names')
   sth.execute
     ...
   sth = nil # forget to finish the statement handle.
   GC.start # <= sth's prepared statement is deallocated by GC.

Sorry, I have not test it. I have no working postgresql database...

diff --git a/lib/dbd/pg/statement.rb b/lib/dbd/pg/statement.rb
index 0271a10..7273e77 100644
--- a/lib/dbd/pg/statement.rb
+++ b/lib/dbd/pg/statement.rb
@@ -10,6 +10,16 @@ class DBI::DBD::Pg::Statement < DBI::BaseStatement

     PG_STMT_NAME_PREFIX = 'ruby-dbi:Pg:'

+    @@id_map = {} # prepared statement's id => db
+
+    @@finalizer_proc = proc do |id|
+        db = @@id_map[id]
+        if db
+            @@id_map.delete(id)
+            db._exec("DEALLOCATE \"#{PG_STMT_NAME_PREFIX + id.to_s}\"")
+        end
+    end
+
     def initialize(db, sql)
         super(db)
         @db  = db
@@ -17,7 +27,7 @@ class DBI::DBD::Pg::Statement < DBI::BaseStatement
         @stmt_name = PG_STMT_NAME_PREFIX + self.object_id.to_s
         @result = nil
         @bindvars = []
-        @prepared = false
+        ObjectSpace.define_finalizer(self, @@finalizer_proc)
     rescue PGError => err
         raise DBI::ProgrammingError.new(err.message)
     end
@@ -125,8 +135,8 @@ class DBI::DBD::Pg::Statement < DBI::BaseStatement
     # finish the statement at a lower level
     def internal_finish
         @result.finish if @result
-        statement_exists = @db._exec("select * from
pg_prepared_statements where name='#{@stmt_name}'")
-        if statement_exists.num_tuples > 0
+        if @@id_map[self.object_id]
+            @@id_map.delete(self.object_id)
             @db._exec("DEALLOCATE \"#{@stmt_name}\"")
         end
     end
@@ -134,14 +144,14 @@ class DBI::DBD::Pg::Statement < DBI::BaseStatement
     # prepare the statement at a lower level.
     def internal_prepare
         if @db["pg_native_binding"]
-            unless @prepared
+            unless @@id_map[self.object_id]
                 @stmt = @db._prepare(@stmt_name, translate_param_markers(@sql))
             end
         else
             internal_finish
             @stmt = @db._prepare(@stmt_name,
DBI::SQL::PreparedStatement.new(DBI::DBD::Pg, @sql).bind(@bindvars))
         end
-        @prepared = true
+        @@id_map[self.object_id] = @db
     end

     # Prepare the given SQL statement, returning its PostgreSQL string

From erik at hollensbe.org  Tue Sep  2 16:37:52 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Tue, 2 Sep 2008 13:37:52 -0700
Subject: [ruby-dbi-users] dbd_pg and deallocating prepared statements
In-Reply-To: <5d847bcd0809012239n48fcb4a7r3124773e34dd68f7@mail.gmail.com>
References: <5d847bcd0808261856p5d77328chf53c14db2edbf3ad@mail.gmail.com>
	<200809011411.16704.erik@hollensbe.org>
	<5d847bcd0809012239n48fcb4a7r3124773e34dd68f7@mail.gmail.com>
Message-ID: <200809021337.52443.erik@hollensbe.org>

On Monday 01 September 2008 22:39:09 KUBO Takehiro wrote:
> The following patch will release prepared statements in GC even though they
> are not finished explicitly.
>    sth = dbh.prepare('select * from names')
>    sth.execute
>      ...
>    sth = nil # forget to finish the statement handle.
>    GC.start # <= sth's prepared statement is deallocated by GC.

Sorry this is so long. KUBO, If you could be so kind as to post this patch on 
the rubyforge patches tracker, it will make it much easier for me to track, as 
I can't integrate it this moment (for more than just the reasons described 
here). 

This is actually a patch I've been wanting to do for a while, but really had 
no good grasp on how to do it. Ruby finalizers are still a bit of a black art 
to me since they aren't really a destructor, but this patch definitely tries 
to use them like one.

I'll read up on finalizers.

However, my biggest concern right now is with the database handle itself, as 
it seems that the dbh instance becomes co-dependent with the statement handle 
for garbage collection. While I certainly don't have a better solution to this 
problem at the moment, the way this is done circumvents the normal chain of 
events:

f.e.:

dbh is created, sth is created. sth does not finish. dbh is disconnected. 
program continues.

at this point there is still a dbh instance (which in every case holds an 
instance of the underlying database driver) that is inoperable.

The statement handle, even when gc'd, would not clean up it's postgresql level 
statement because it would be unable to do so, since the database is 
disconnected. Now, PostgreSQL may do this for us automatically, but I do think 
it's worth considering that it doesn't, or a similar pattern in another DBD 
would render this problem, at least for the sake of this argument.

I should clarify that one of my long-term goals with Ruby/DBI is to enforce 
policy more in DBI itself and alleviate DBDs from having to implement policy 
unless absolutely necessary; one of the largest sources of bug reports is DBI 
*saying* one thing and the DBD *doing* something else, which DBI does not 
properly prevent.

So, while I think a finalizer used in this fashion would be a good way to 
handle this problem, I think there are higher-level concerns. 0.4.0 currently 
expects the user to finish all their statement handles, and I don't think 
changing this functionality in one DBD solves the problem.

So, that brings us to 0.6.0. 

One thing that *all* DBDs would benefit from is a DatabaseHandle modification 
to #prepare (and friends) that stores the newly created StatementHandle before 
passing it off. At this point, we have all the information required to finish 
handles before disconnection happens. (In the event of premature termination, 
we can't do much anyways,)

Also, if we did this finalizer at a higher level (StatementHandle or maybe 
even BaseStatement) all DBDs would be able to leverage this functionality, 
allowing DBI to manage the process up to the point that the DBD needs to get 
involved. 

It also leaves open the opportunity for DatabaseHandle finalizers that operate 
in a similar fashion, and preserves the chain of creation and destruction 
without side effects. It's also a consistent process that benefits all DBI 
users.

So, to boil this down I think your idea is splendid, and I want to expand on 
it, but I don't think it's the full solution, nor is it something that can be 
reasonably considered for a patchlevel release. What do you think?

-Erik

From erik at hollensbe.org  Tue Sep  2 22:21:48 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Tue, 2 Sep 2008 19:21:48 -0700
Subject: [ruby-dbi-users] dbd_pg and deallocating prepared statements
In-Reply-To: <200809011411.16704.erik@hollensbe.org>
References: <5d847bcd0808261856p5d77328chf53c14db2edbf3ad@mail.gmail.com>
	<200809011411.16704.erik@hollensbe.org>
Message-ID: <200809021921.48277.erik@hollensbe.org>

On Monday 01 September 2008 14:11:16 Erik Hollensbe wrote:
> This bug (and a type handling bug in mysql) have been fixed in the recent
> git push. If anyone has some spare time and would just like to run the test
> suite which should reveal any issues in either of these changes, it'll ease
> my mind to release it today.

These bugs and a fix to the ODBC DBD have been released as gems, and are 
available on rubyforge now. They will likely be available via the 'gem' 
command in the next few hours.

-Erik

From kubo at jiubao.org  Thu Sep  4 02:18:46 2008
From: kubo at jiubao.org (KUBO Takehiro)
Date: Thu, 4 Sep 2008 15:18:46 +0900
Subject: [ruby-dbi-users] dbd_pg and deallocating prepared statements
In-Reply-To: <200809021337.52443.erik@hollensbe.org>
References: <5d847bcd0808261856p5d77328chf53c14db2edbf3ad@mail.gmail.com>
	<200809011411.16704.erik@hollensbe.org>
	<5d847bcd0809012239n48fcb4a7r3124773e34dd68f7@mail.gmail.com>
	<200809021337.52443.erik@hollensbe.org>
Message-ID: <5d847bcd0809032318p6afbc57cjf6f8bfabe22306f7@mail.gmail.com>

Hi,

On Wed, Sep 3, 2008 at 5:37 AM, Erik Hollensbe <erik at hollensbe.org> wrote:
> On Monday 01 September 2008 22:39:09 KUBO Takehiro wrote:
>> The following patch will release prepared statements in GC even though they
>> are not finished explicitly.
>>    sth = dbh.prepare('select * from names')
>>    sth.execute
>>      ...
>>    sth = nil # forget to finish the statement handle.
>>    GC.start # <= sth's prepared statement is deallocated by GC.
>
> Sorry this is so long. KUBO, If you could be so kind as to post this patch on
> the rubyforge patches tracker, it will make it much easier for me to track, as
> I can't integrate it this moment (for more than just the reasons described
> here).

Okay. I'll submit it.

> f.e.:
>
> dbh is created, sth is created. sth does not finish. dbh is disconnected.
> program continues.
>
> at this point there is still a dbh instance (which in every case holds an
> instance of the underlying database driver) that is inoperable.
>
> The statement handle, even when gc'd, would not clean up it's postgresql level
> statement because it would be unable to do so, since the database is
> disconnected. Now, PostgreSQL may do this for us automatically, but I do think
> it's worth considering that it doesn't, or a similar pattern in another DBD
> would render this problem, at least for the sake of this argument.

PostgreSQL does it automatically.

  http://www.postgresql.org/docs/8.3/static/sql-deallocate.html
    If you do not explicitly deallocate a prepared statement, it is
    deallocated when the session ends.

IMO, if server side statement resources are not freed even though the
connection is closed, it is a bug of the DBMS. If such DBMS exists, it
should be solved by native database drivers (for example: ruby-oci8,
ruby-pg, MySQL/Ruby, etc.) for who use them directly. Once the native
drivers solve the problem, dbi has no need to care about it.

The only exception is PostgreSQL's server side prepared statement.
As for other native drivers, statement handle instances correspond to
server side statement resources. When a dbi statement handle is GC'd,
the native driver's statement handle will be GC'd in a short time
and the associated server side statement is also freed. But
PostgreSQL's server side prepared statement is identified by a
name. ruby-pg has no way to know whether the name will be used later
or not. ruby-pg users may use another String instance which has same
name.

> Also, if we did this finalizer at a higher level (StatementHandle or maybe
> even BaseStatement) all DBDs would be able to leverage this functionality,
> allowing DBI to manage the process up to the point that the DBD needs to get
> involved.
>
> It also leaves open the opportunity for DatabaseHandle finalizers that operate
> in a similar fashion, and preserves the chain of creation and destruction
> without side effects. It's also a consistent process that benefits all DBI
> users.

Another reason: It is easy for C extension libraries such as native
database drivers to use a finalizer, but not for pure ruby libraries
such as ruby-dbi. As for C extension libraries, it is just adding a C
callback function. The function called just before the object is
GC'd. It means that the object is alive at the point of time. But a
ruby proc object which is registerd by ObjectSpace.define_finalizer
can only know the GC'd object's id. The object itself has been already
GC'd. What you want to do is very difficult.

> So, to boil this down I think your idea is splendid, and I want to expand on
> it, but I don't think it's the full solution, nor is it something that can be
> reasonably considered for a patchlevel release. What do you think?

I think ruby-dbi has no need to use finalizers except dbd_pg statement
handle. It changes dbd_pg only. It is considered for a patchlevel
release. But I have another issue of dbd_pg, which will be posted to a
new thread.

From erik at hollensbe.org  Thu Sep  4 06:23:24 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Thu, 4 Sep 2008 03:23:24 -0700
Subject: [ruby-dbi-users] dbd_pg and deallocating prepared statements
In-Reply-To: <5d847bcd0809032318p6afbc57cjf6f8bfabe22306f7@mail.gmail.com>
References: <5d847bcd0808261856p5d77328chf53c14db2edbf3ad@mail.gmail.com>
	<200809021337.52443.erik@hollensbe.org>
	<5d847bcd0809032318p6afbc57cjf6f8bfabe22306f7@mail.gmail.com>
Message-ID: <200809040323.24889.erik@hollensbe.org>

On Wednesday 03 September 2008 23:18:46 KUBO Takehiro wrote:
> PostgreSQL does it automatically.
>
>   http://www.postgresql.org/docs/8.3/static/sql-deallocate.html
>     If you do not explicitly deallocate a prepared statement, it is
>     deallocated when the session ends.
>
> IMO, if server side statement resources are not freed even though the
> connection is closed, it is a bug of the DBMS. If such DBMS exists, it
> should be solved by native database drivers (for example: ruby-oci8,
> ruby-pg, MySQL/Ruby, etc.) for who use them directly. Once the native
> drivers solve the problem, dbi has no need to care about it.
>
> The only exception is PostgreSQL's server side prepared statement.
> As for other native drivers, statement handle instances correspond to
> server side statement resources. When a dbi statement handle is GC'd,
> the native driver's statement handle will be GC'd in a short time
> and the associated server side statement is also freed. But
> PostgreSQL's server side prepared statement is identified by a
> name. ruby-pg has no way to know whether the name will be used later
> or not. ruby-pg users may use another String instance which has same
> name.

Even though this is the case, the problem still exists that the statement will 
attempt to deallocate on a disconnected database. While that check can be done 
in the finalizers, I'd like to see support at a higher level for this kind of 
operation; statements shouldn't exist when corresponding database handles 
don't, and so on.

The current situation (where Pg is the only DBD distributed with DBI that does 
server-side prepared statements) is the result of poor support in the DBDs 
that work against databases with prepared statement support.

> Another reason: It is easy for C extension libraries such as native
> database drivers to use a finalizer, but not for pure ruby libraries
> such as ruby-dbi. As for C extension libraries, it is just adding a C
> callback function. The function called just before the object is
> GC'd. It means that the object is alive at the point of time. But a
> ruby proc object which is registerd by ObjectSpace.define_finalizer
> can only know the GC'd object's id. The object itself has been already
> GC'd. What you want to do is very difficult.

I still have to read up on this; so I apologize, but I'm answering off the 
cuff. Is this really DBI's problem at this point, or a poor separation of 
powers on the part of the DBD design?

I guess I'm bringing this up because the way most of the DBDs are designed is 
to keep at least two layers between the C code and the end-user classes... 
BaseDatabase and DatabaseHandle respectively. You can see that in your 
solution that spurred this conversation: at no point are any dangling PGresult 
handles being cleaned up, or even recognized in the finalizers, and that's OK, 
because ruby (and likely ruby-pg) will take care of that for us naturally.

> > So, to boil this down I think your idea is splendid, and I want to expand
> > on it, but I don't think it's the full solution, nor is it something that
> > can be reasonably considered for a patchlevel release. What do you think?
>
> I think ruby-dbi has no need to use finalizers except dbd_pg statement
> handle. It changes dbd_pg only. It is considered for a patchlevel
> release. But I have another issue of dbd_pg, which will be posted to a
> new thread.

Again, sorry for being a windbag, but I really understand your reasoning and 
want to make it undeniably clear why I disagree with the timing and think it's 
best to wait.

Using finalizers is not the issue I'm attempting to address here, it's an 
implementation detail of the problem, which is that #finish must be called 
explicitly right now, and both of us agree that it shouldn't be necessary. 
What we disagree on is where this should be handled, or more precisely, what's 
the path that's taken to define these handlers.

Obviously, as your patch demonstrates and we both know, the only thing keeping 
this behavior from happening with the Pg driver is for me to integrate your 
patch.

However, DBI claims to have a consistent interface to SQL and in many cases it 
doesn't, and a good 90% of the reasons for this is because different DBDs do 
the same things different ways, and DBI allows it. If my words don't help, 
take a look at the DBD test suite, which grows one test at a time per bug, and 
has become decently sized since I built it against the original CVS repository 
checkout I started with. DBI proper, in contrast, has remained relatively 
untouched; the only test modifications (and bugs) have been related to feature 
additions. 

You can also see this in the bug reports, where most users are expecting 
something that DBI claims it does and it clearly does not, because the user is 
on MySQL or SQLite and the only place that works properly is on Pg. Normally, 
the problem has nothing to do with the databases themselves, but the DBD that 
assumes too much interface. I assert this is DBI's problem.

Nothing is requiring DBDs to define finalizers, just if they do to use the 
interface provided. The goal is to ensure #finish and #disconnect work 
consistently; at that point it's the DBDs problem to define an appropriate 
finalizer ("none" may be appropriate) as DBI has met its part of the contract.

If we want to contrast this with Perl's DBI, there are several (simpler) DBDs 
that define little more than a few hash entries and a fetch method. That can't 
be done right now (or get remotely close to it) in Ruby/DBI, and I consider 
that a design fault.

Another way of looking at it: DBDs should have tunnel vision and DBI maintains 
the big picture.

That said, I appreciate your input and effort on this, and I can guarantee 
that as soon as I can find enough time to move forward on 0.6.0, this will be 
the first thing I tackle.

-Erik

From erik at hollensbe.org  Sat Sep  6 06:20:52 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Sat, 6 Sep 2008 03:20:52 -0700
Subject: [ruby-dbi-users] git changes
Message-ID: <200809060320.52517.erik@hollensbe.org>

I'm a little resigned to do this, especially now that people are using the git 
repositories on a wider scale.

However, the independent development of DBI and the DBDs pretty much requires 
this, and 0.6.0 is locked from moving forward until some progress is made on 
it.

Sometime this week I'll be breaking the DBI repository into pieces which 
represent the four supported DBDs and DBI. I will be forced to move files 
around before I do this that will be unable to be pushed until the work is 
complete.

If you have unfinished patches or other local modifications, pay attention to 
the following changes and timeline:

1) Patches to get DBI building without rubygems will be created; DBI 0.4.1 
will be released
2) DBI repository will be slightly restructured to take advantage of 
submodules, this will initially be contained to the lib/dbd directory, but 
build tools will be modified as well.
3) DBD repos will be created and pushed
4) DBI repo (with submodules) will be pushed
5) Branches will be created in all repos that echo our current development 
strategy; 'master' for maintenance patching for release versions, and 
'development' for features/architecture changes that will be included in the 
next release.
6) scripts will be created to sync DBI and the submodules, hopefully easing 
the configuration/management burden on casual users.
7) at *some point*, DBD repositories will be self-hosting and truly 
independent (their own test suites, etc). This is not a priority and 0.6.0 
could likely come long before that.

I'm having trouble coming up with a better situation than this, and have asked 
several communities on how to solve it, and basically got 'it sucks, but it 
sounds like you need submodules' as an answer from all of them.

In other words, I would *love* to hear better solutions. Please don't hold 
back.

Thanks,

-Erik

From kubo at jiubao.org  Sat Sep  6 09:59:54 2008
From: kubo at jiubao.org (KUBO Takehiro)
Date: Sat, 6 Sep 2008 22:59:54 +0900
Subject: [ruby-dbi-users] dbd_pg and deallocating prepared statements
In-Reply-To: <200809040323.24889.erik@hollensbe.org>
References: <5d847bcd0808261856p5d77328chf53c14db2edbf3ad@mail.gmail.com>
	<200809021337.52443.erik@hollensbe.org>
	<5d847bcd0809032318p6afbc57cjf6f8bfabe22306f7@mail.gmail.com>
	<200809040323.24889.erik@hollensbe.org>
Message-ID: <5d847bcd0809060659q3553cf5dr47cfe876be1951f6@mail.gmail.com>

Sorry so long.

> Even though this is the case, the problem still exists that the statement will
> attempt to deallocate on a disconnected database. While that check can be done
> in the finalizers, I'd like to see support at a higher level for this kind of
> operation; statements shouldn't exist when corresponding database handles
> don't, and so on.

When deallocating on a disconnected database, "not connected"
exception is raised. But an exception in GC is ignored. So nobody need
to rescue it. I had to add a comment to the patch about this, but I
omitted it...

> The current situation (where Pg is the only DBD distributed with DBI that does
> server-side prepared statements) is the result of poor support in the DBDs
> that work against databases with prepared statement support.

Ruby-oci8 also supports prepared statement, but it manages statement
handles by itself. IMO, MySQL, DB2, ODBC, SQL Server, SQLite are also
similar. PostgreSQL's prepared statement is quite unique.

> Using finalizers is not the issue I'm attempting to address here, it's an
> implementation detail of the problem, which is that #finish must be called
> explicitly right now, and both of us agree that it shouldn't be necessary.
> What we disagree on is where this should be handled, or more precisely, what's
> the path that's taken to define these handlers.

I think that it should be done when more than two dbd drivers
(including currently unsupported) are expected to need finalizers.
I'm not against you. I'm just concerned that if no dbd except dbd_pg
use it, it makes dbi complex more than needs.

From databaseofgrace.5.pistos at geoshell.com  Sat Sep  6 09:11:08 2008
From: databaseofgrace.5.pistos at geoshell.com (databaseofgrace.5.pistos at geoshell.com)
Date: Sat, 6 Sep 2008 09:11:08 -0400
Subject: [ruby-dbi-users] git changes
In-Reply-To: <200809060320.52517.erik@hollensbe.org>
References: <200809060320.52517.erik@hollensbe.org>
Message-ID: <6c9d9ef0809060611h510f236ei84c940e054f02f56@mail.gmail.com>

2008/9/6 Erik Hollensbe - erik at hollensbe.org <+databaseofgrace+pistos>
7) at *some point*, DBD repositories will be self-hosting and truly
> independent (their own test suites, etc). This is not a priority and 0.6.0
> could likely come long before that.
>
> I'm having trouble coming up with a better situation than this, and have asked
> several communities on how to solve it, and basically got 'it sucks, but it
> sounds like you need submodules' as an answer from all of them.
>
> In other words, I would *love* to hear better solutions. Please don't hold
> back.

Erik: My initial thought is how about you fork or clone the existing
single repo several times, once for each DBD, then in the clones, take
a light sabre and lop off everything but the DBD files and dirs you
want to keep for that one clone, and then setup a new remote (i.e.
github fork) with an appropriate name (e.g. dbd-pg).  That way, you
keep the full history to date, but moving forward, each
properly-renamed clone contains only the DBD it's meant to, or in the
case of DBI, DBI alone.

How about that?


-- 
Pistos
http://blog.purepistos.net

From erik at hollensbe.org  Sat Sep  6 15:08:58 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Sat, 6 Sep 2008 12:08:58 -0700
Subject: [ruby-dbi-users] git changes
In-Reply-To: <6c9d9ef0809060611h510f236ei84c940e054f02f56@mail.gmail.com>
References: <200809060320.52517.erik@hollensbe.org>
	<6c9d9ef0809060611h510f236ei84c940e054f02f56@mail.gmail.com>
Message-ID: <200809061208.58841.erik@hollensbe.org>

On Saturday 06 September 2008 06:11:08 databaseofgrace.5.pistos at geoshell.com 
wrote:
> >
> > In other words, I would *love* to hear better solutions. Please don't
> > hold back.
>
> Erik: My initial thought is how about you fork or clone the existing
> single repo several times, once for each DBD, then in the clones, take
> a light sabre and lop off everything but the DBD files and dirs you
> want to keep for that one clone, and then setup a new remote (i.e.
> github fork) with an appropriate name (e.g. dbd-pg).  That way, you
> keep the full history to date, but moving forward, each
> properly-renamed clone contains only the DBD it's meant to, or in the
> case of DBI, DBI alone.

Unless I misunderstand you, that's how I plan to do it, but I will do all the 
pruning before I push them.

-Erik

From kubo at jiubao.org  Sat Sep  6 22:05:34 2008
From: kubo at jiubao.org (KUBO Takehiro)
Date: Sun, 7 Sep 2008 11:05:34 +0900
Subject: [ruby-dbi-users] [dbd_pg] Prepared statement performance
Message-ID: <5d847bcd0809061905q198ed64agc1fe708900c12578@mail.gmail.com>

I looked in manuals and source codes of libpq, ruby-pg, perl dbd::pg
and postgrsql jdbc driver. But I have not run it. I may misunderstand
the behavior.

Server side prepared statements are not always good for performance.
It depend on the database archtecture. For example. On Oracle, all
statement's execution plans are cached in server side shared memory
and reused by all other connections. This means that the folloing
steps are skipped if the SQL is on the cache.
 * parse the SQL statement.
 * make possible execution (access) plans.
 * choose one by comparing estimated costs

But postgresql's SQL statements are not cached automatically. Database
client application must manage it explicitly by a name of the
statement, and it is reused only when the name is reused explicitly.

IMO, almost dbi applications have a negative impact rather than
benefit.

Case 1:
   dbh.execute('update emp set sal = sal + 10 where empno = 1')

This dbi code executes the following ruby-pg code internally.
   conn.prepare('ruby-dbi:Pg:1234', 'update emp set sal = sal + 10
where empno = 1')
   conn.exec_prepared('ruby-dbi:Pg:1234')
   conn.exec('DEALLOCATE "ruby-dbi:Pg:1234"')

It needs three network round trips. If dbd_pg didn't use prepared
statements as follows, it needed only one round trip.
   conn.exec('update emp set sal = sal + 10 where empno = 1')

This is not a small impact if the application runs in DMZ and the
database server is behind the firewall.


Case 2:
   dbh.execute('update emp set sal = sal + 10 where empno = ?', 1)
   dbh.execute('update emp set sal = sal + 10 where empno = ?', 2)

This executes the following code internally.
   conn.prepare('ruby-dbi:Pg:1234', 'update emp set sal = sal + 10
where empno = $1')
   conn.exec_prepared('ruby-dbi:Pg:1234', [1])
   conn.exec('DEALLOCATE "ruby-dbi:Pg:1234"')
   conn.prepare('ruby-dbi:Pg:1235', 'update emp set sal = sal + 10
where empno = $1')
   conn.exec_prepared('ruby-dbi:Pg:1235', [2])
   conn.exec('DEALLOCATE "ruby-dbi:Pg:1235"')

It receive no benefit of prepared statements. The sql statements are
exactly same but the statement names are different. Even though the
name are accidentaly same (same object id), the first sql's execution
plan has been already deallocated.

It needs six network round trips. If dbd_pg didn't use prepared
statements as follows, it needed only two round trips.
   conn.exec('update emp set sal = sal + 10 where empno = $1', [1])
   conn.exec('update emp set sal = sal + 10 where empno = $1', [2])


Case 3:
   sth = dbh.prepare('update emp set sal = sal + 10 where empno = ?')
   1.upto 100 do |i|
     sth.execute(i)
   end
   sth.finish

This executes the following code internally.
   conn.prepare('ruby-dbi:Pg:1234', 'update emp set sal = sal + 10
where empno = $1')
   1.upto 100 do |i|
     conn.exec_prepared('ruby-dbi:Pg:1234', [i])
   end
   conn.exec('DEALLOCATE "ruby-dbi:Pg:1234"')

This is a good case. The sql statement is parsed and the execution
plan is made only once. The plan is reused 100 times. If ruby-dbi
didn't use a prepared statement, the posgres server would parse 100
sql statements and make 100 execution plans.


In the three cases, the last one is the only case which has a benefit
of prepared statement. But case 3 is rare when the application is for
transaction system. Majority code will issue queries such as case 1
or case 2.

IMO, dbd_pg should not use server side prepared statements by default.
They should be used only when (1) it is requested or (2) dbd_pg
consider that now is the time to use them.

(1) add an attribute "pg_server_prepare" as Perl DBD::Pg does.
    (pg_server_prepare is enabled by default on perl dbd...)

    http://search.cpan.org/~turnstep/DBD-Pg/Pg.pm#prepare

  For example:

    DBI.connect("dbi:Pg:database=#{dbname};pg_serer_prepare=true", ...)
      => set default value of the connection.

    dbh.prepare(sql_statement, {:pg_server_prepare => true})
      => use server prepare.

    dbh.prepare(sql_statement, {:pg_server_prepare => false})
      => don't use server prepare.

    dbh.prepare(sql_statement)
      => depend on default value of the connection.

(2) enable server side prepared statemetn when the sth is executed 5
    (or user-specified number) times as postgresql jdbc driver does.

      http://jdbc.postgresql.org/documentation/81/server-prepare.html

  For example:

    sth = dbh.prepare('update emp set sal = sal + 10 where empno = ?')
    sth.execute(1)
    # => conn.exec('update emp set sal = sal + 10 where empno = $1', [1])

    sth.execute(2)
    # => conn.exec('update emp set sal = sal + 10 where empno = $1', [2])

    sth.execute(3)
    # => conn.exec('update emp set sal = sal + 10 where empno = $1', [3])

    sth.execute(4)
    # => conn.exec('update emp set sal = sal + 10 where empno = $1', [4])

    sth.execute(5)
    # => conn.exec('update emp set sal = sal + 10 where empno = $1', [5])

    sth.execute(6)
    # reach the threshold. enable server side prepare.
    # => conn.prepare('ruby-dbi:Pg:1234', 'update emp set sal = sal +
10 where empno = $1')
    #    conn.exec_prepared('ruby-dbi:Pg:1234', [6])

    sth.execute(7)
    # => conn.exec_prepared('ruby-dbi:Pg:1234', [7])

      .....

    sth.finish
    # => conn.exec('DEALLOCATE "ruby-dbi:Pg:1234"')

I prefer the latter because ruby-dbi users have no need to care whether
server side prepared statements should be useed or not.

Well, I'll add this issue to 'Feature Requests' also.

From erik at hollensbe.org  Sun Sep  7 03:29:05 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Sun, 7 Sep 2008 00:29:05 -0700
Subject: [ruby-dbi-users] [dbd_pg] Prepared statement performance
In-Reply-To: <5d847bcd0809061905q198ed64agc1fe708900c12578@mail.gmail.com>
References: <5d847bcd0809061905q198ed64agc1fe708900c12578@mail.gmail.com>
Message-ID: <200809070029.05322.erik@hollensbe.org>

On Saturday 06 September 2008 19:05:34 KUBO Takehiro wrote:
> But postgresql's SQL statements are not cached automatically. Database
> client application must manage it explicitly by a name of the
> statement, and it is reused only when the name is reused explicitly.

Sounds great, makes perfect sense. Thank you for pointing this out. 
Performance issues aren't my focus at this point, and if you're willing to 
tackle them I'm happy to accept patches.

-Erik


From erik at hollensbe.org  Sun Sep  7 03:49:04 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Sun, 7 Sep 2008 00:49:04 -0700
Subject: [ruby-dbi-users] [dbd_pg] Prepared statement performance
In-Reply-To: <200809070029.05322.erik@hollensbe.org>
References: <5d847bcd0809061905q198ed64agc1fe708900c12578@mail.gmail.com>
	<200809070029.05322.erik@hollensbe.org>
Message-ID: <200809070049.04834.erik@hollensbe.org>

On Sunday 07 September 2008 00:29:05 Erik Hollensbe wrote:
> On Saturday 06 September 2008 19:05:34 KUBO Takehiro wrote:
> > But postgresql's SQL statements are not cached automatically. Database
> > client application must manage it explicitly by a name of the
> > statement, and it is reused only when the name is reused explicitly.
>
> Sounds great, makes perfect sense. Thank you for pointing this out.
> Performance issues aren't my focus at this point, and if you're willing to
> tackle them I'm happy to accept patches.

Oh, I just realized... This may butt heads with the pg_native_binds option, 
which uses server-side bound parameters as well as the prepared statements 
(when it's off, it uses the DBI quoter, which is a little more flexible), so 
if you do end up patching this, you may need to be aware that depending on the 
situation, these options may need to be co-dependent. The majority of the 
relevant code is in lib/dbd/pg/statement.rb.

Thanks again,

-Erik

From kubo at jiubao.org  Sun Sep  7 10:14:06 2008
From: kubo at jiubao.org (KUBO Takehiro)
Date: Sun, 7 Sep 2008 23:14:06 +0900
Subject: [ruby-dbi-users] [dbd_pg] Prepared statement performance
In-Reply-To: <200809070049.04834.erik@hollensbe.org>
References: <5d847bcd0809061905q198ed64agc1fe708900c12578@mail.gmail.com>
	<200809070029.05322.erik@hollensbe.org>
	<200809070049.04834.erik@hollensbe.org>
Message-ID: <5d847bcd0809070714q42645471x4f49b985f97661be@mail.gmail.com>

On Sun, Sep 7, 2008 at 4:49 PM, Erik Hollensbe <erik at hollensbe.org> wrote:
> On Sunday 07 September 2008 00:29:05 Erik Hollensbe wrote:
>> On Saturday 06 September 2008 19:05:34 KUBO Takehiro wrote:
>> > But postgresql's SQL statements are not cached automatically. Database
>> > client application must manage it explicitly by a name of the
>> > statement, and it is reused only when the name is reused explicitly.
>>
>> Sounds great, makes perfect sense. Thank you for pointing this out.
>> Performance issues aren't my focus at this point, and if you're willing to
>> tackle them I'm happy to accept patches.

Well sorry, I'm not willing to do it for now. My priority is (1)
ruby-oci8 2.0, which
should be released before ruby 1.9.1, (2) dbd::oci8. In addition I'm not certain
whether the feature should be added to dbd::pg or ruby-pg.

> Oh, I just realized... This may butt heads with the pg_native_binds option,
> which uses server-side bound parameters as well as the prepared statements
> (when it's off, it uses the DBI quoter, which is a little more flexible), so
> if you do end up patching this, you may need to be aware that depending on the
> situation, these options may need to be co-dependent. The majority of the
> relevant code is in lib/dbd/pg/statement.rb.

Yes. Usable combinations are:
1. native binding and server-side prepared statement
2. native binding and prepared statement emulation
3. binding emulation and prepared statement emulation

The next is useless because it can be used only when the bind values are not
changed for each execution.
4. binding emulation and server-side prepared statement

From jarl at gavia.dk  Mon Sep 29 14:28:04 2008
From: jarl at gavia.dk (Jarl Friis)
Date: Mon, 29 Sep 2008 20:28:04 +0200
Subject: [ruby-dbi-users] Proposing patch for sqlite3 driver
Message-ID: <m3wsguu7rf.fsf@zeus.intra.softace.dk>

I am proposing the patch contributed in bug#22132 to handle the sqlite3
type system more closely.

comments?

Jarl

-- 
Jarl Friis
Gavia Technologies ApS
Om?gade 8, 2.sal
2100 K?benhavn ?.
Denmark
Phone:  +45 26 13 20 90
E-mail: jarl at gavia.dk
LinkedIn: www.linkedin.com/in/jarlfriis


From grzm at seespotcode.net  Mon Sep 29 15:31:55 2008
From: grzm at seespotcode.net (Michael Glaesemann)
Date: Mon, 29 Sep 2008 15:31:55 -0400
Subject: [ruby-dbi-users] DBI 0.4.0 & DBD::Pg gem installation issues
Message-ID: <103B9BB3-DBB3-44DD-9180-D33695289779@seespotcode.net>

I've installed ruby via yum on RHEL5, and am having issues with  
getting DBD::Pg to work.

Here's a test script I've been using for debugging:

[grzm at db05 ~]$ cat test_dbd_pg.rb
#!/usr/bin/ruby -w

require 'rubygems'
require 'dbi'

db_host = 'localhost'
db_port = '5432'
db_name = 'production'
db_user = 'postgres'

query = 'SELECT version()';
dbh = DBI.connect(%(dbi:Pg:#{db_name}:#{db_host}:#{db_port}), db_user)  
do |dbh|
   sth = dbh.execute(query)
   sth.each do |row|
      puts %Q(#{row['version']})
   end
end
[grzm at db05 ~]$ ruby test_dbd_pg.rb
/usr/lib64/ruby/gems/1.8/gems/dbi-0.4.0/lib/dbi.rb:286:in  
`load_driver': Unable to load driver 'Pg' (DBI::InterfaceError)
	from /usr/lib/ruby/1.8/monitor.rb:238:in `synchronize'
	from /usr/lib64/ruby/gems/1.8/gems/dbi-0.4.0/lib/dbi.rb:236:in  
`load_driver'
	from /usr/lib64/ruby/gems/1.8/gems/dbi-0.4.0/lib/dbi.rb:154:in  
`_get_full_driver'
	from /usr/lib64/ruby/gems/1.8/gems/dbi-0.4.0/lib/dbi.rb:139:in  
`connect'
	from test_dbd_pg.rb:12

Just confirming that the connection parameters should work (if the  
script got to that point).

[grzm at db05 ~]$ /usr/local/pgsql/bin/psql -h localhost -p 5432 -U  
postgres production
Welcome to psql 8.3.3, the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
        \h for help with SQL commands
        \? for help with psql commands
        \g or terminate with semicolon to execute query
        \q to quit

production=# \q

And a quick test in irb showing an error when just requiring pg:

[grzm at db05 ~]$ irb
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'dbi'
=> true
irb(main):003:0> gem 'pg'
=> true
irb(main):005:0> require 'pg'
LoadError: libpq.so.5: cannot open shared object file: No such file or  
directory - /usr/lib64/ruby/gems/1.8/gems/pg-0.7.9.2008.08.17/lib/pg.so
	from /usr/lib64/ruby/gems/1.8/gems/pg-0.7.9.2008.08.17/lib/pg.so
	from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in  
`require'
	from (irb):5
irb(main):006:0>
[grzm at db05 ~]$ ls -la /usr/lib64/ruby/gems/1.8/gems/ 
pg-0.7.9.2008.08.17/lib/pg.so
-rwxr-xr-x 1 root root 131132 Sep 19 12:19 /usr/lib64/ruby/gems/1.8/ 
gems/pg-0.7.9.2008.08.17/lib/pg.so

Any help or further debugging pointers would be much appreciated.

Michael Glaesemann
grzm seespotcode net




From erik at hollensbe.org  Tue Sep 30 01:41:55 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Mon, 29 Sep 2008 22:41:55 -0700
Subject: [ruby-dbi-users] DBI 0.4.0 & DBD::Pg gem installation issues
In-Reply-To: <103B9BB3-DBB3-44DD-9180-D33695289779@seespotcode.net>
References: <103B9BB3-DBB3-44DD-9180-D33695289779@seespotcode.net>
Message-ID: <200809292241.56002.erik@hollensbe.org>

On Monday 29 September 2008 12:31:55 Michael Glaesemann wrote:
> irb(main):005:0> require 'pg'
> LoadError: libpq.so.5: cannot open shared object file: No such file or
> directory - /usr/lib64/ruby/gems/1.8/gems/pg-0.7.9.2008.08.17/lib/pg.so
> 	from /usr/lib64/ruby/gems/1.8/gems/pg-0.7.9.2008.08.17/lib/pg.so
> 	from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
> `require'
> 	from (irb):5
> irb(main):006:0>
> [grzm at db05 ~]$ ls -la /usr/lib64/ruby/gems/1.8/gems/
> pg-0.7.9.2008.08.17/lib/pg.so
> -rwxr-xr-x 1 root root 131132 Sep 19 12:19 /usr/lib64/ruby/gems/1.8/
> gems/pg-0.7.9.2008.08.17/lib/pg.so
>
> Any help or further debugging pointers would be much appreciated.

It looks like the 'pg' library did not install correctly, or for some reason 
it can't dlopen libpq. I just throw bugs in their direction occasionally, and 
do not work on the project itself.

http://rubyforge.org/projects/ruby-pg should be the place to report such 
things; be certain to give them the output of ldd and a test case that 
doesn't involve DBI, as it seems DBI's shoddy error reporting is the only 
issue here (which has a long, outstanding ticket that I should get around to 
resolving).

Hope that helps,

-Erik

From erik at hollensbe.org  Tue Sep 30 01:42:38 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Mon, 29 Sep 2008 22:42:38 -0700
Subject: [ruby-dbi-users] Proposing patch for sqlite3 driver
In-Reply-To: <m3wsguu7rf.fsf@zeus.intra.softace.dk>
References: <m3wsguu7rf.fsf@zeus.intra.softace.dk>
Message-ID: <200809292242.39040.erik@hollensbe.org>

On Monday 29 September 2008 11:28:04 Jarl Friis wrote:
> I am proposing the patch contributed in bug#22132 to handle the sqlite3
> type system more closely.
>
> comments?

Jarl,

I have a lot of things on my plate, and it will be unlikely that I will be 
able to get to this patch until next weekend. I will happily reply to this in 
more detail then.

Thanks,

-Erik

From erik at hollensbe.org  Tue Sep 30 01:58:09 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Mon, 29 Sep 2008 22:58:09 -0700
Subject: [ruby-dbi-users] DBI 0.4.0 & DBD::Pg gem installation issues
In-Reply-To: <200809292241.56002.erik@hollensbe.org>
References: <103B9BB3-DBB3-44DD-9180-D33695289779@seespotcode.net>
	<200809292241.56002.erik@hollensbe.org>
Message-ID: <200809292258.09149.erik@hollensbe.org>

On Monday 29 September 2008 22:41:55 Erik Hollensbe wrote:
> On Monday 29 September 2008 12:31:55 Michael Glaesemann wrote:
> >
> > Any help or further debugging pointers would be much appreciated.
>
> It looks like the 'pg' library did not install correctly, or for some
> reason it can't dlopen libpq. I just throw bugs in their direction
> occasionally, and do not work on the project itself.
>
> http://rubyforge.org/projects/ruby-pg should be the place to report such
> things; be certain to give them the output of ldd and a test case that
> doesn't involve DBI, as it seems DBI's shoddy error reporting is the only
> issue here (which has a long, outstanding ticket that I should get around
> to resolving).

Eh, to elaborate:

ruby -rubygems -e 'require "pg"' should be enough to prove your failure case.

-Erik

From jarl at gavia.dk  Tue Sep 30 03:52:34 2008
From: jarl at gavia.dk (Jarl Friis)
Date: Tue, 30 Sep 2008 09:52:34 +0200
Subject: [ruby-dbi-users] Proposing patch for sqlite3 driver
In-Reply-To: <200809292242.39040.erik@hollensbe.org> (Erik Hollensbe's
	message of "Mon, 29 Sep 2008 22:42:38 -0700")
References: <m3wsguu7rf.fsf@zeus.intra.softace.dk>
	<200809292242.39040.erik@hollensbe.org>
Message-ID: <m3bpy6gjel.fsf@zeus.intra.softace.dk>

Erik Hollensbe <erik at hollensbe.org> writes:

> On Monday 29 September 2008 11:28:04 Jarl Friis wrote:
> 
> Jarl,
>
> I have a lot of things on my plate, and it will be unlikely that I
> will be able to get to this patch until next weekend. I will happily
> reply to this in more detail then.

Sounds good enough for me, please take my newest proposal (just sent
on this list).

I hope you also get the opportunity to have a look at the DBD:Pg
patch. If you want trigger that defect you need to configure the
postgres server to use non-US datestyle, in a european installation
/var/lib/pgsql/data/postgresql.conf will typically contain "datestyle
= 'iso, dmy'", but in a US installation I guess this will be
"datestyle = 'iso, mdy'". This is why the PG tests fails on my
european configuration.

Jarl


From grzm at seespotcode.net  Tue Sep 30 08:39:05 2008
From: grzm at seespotcode.net (Michael Glaesemann)
Date: Tue, 30 Sep 2008 08:39:05 -0400
Subject: [ruby-dbi-users] DBI 0.4.0 & DBD::Pg gem installation issues
In-Reply-To: <200809292258.09149.erik@hollensbe.org>
References: <103B9BB3-DBB3-44DD-9180-D33695289779@seespotcode.net>
	<200809292241.56002.erik@hollensbe.org>
	<200809292258.09149.erik@hollensbe.org>
Message-ID: <7FE0D2CD-EAFC-4136-9D20-9E44B7483F55@seespotcode.net>


On Sep 30, 2008, at 1:58 , Erik Hollensbe wrote:

> On Monday 29 September 2008 22:41:55 Erik Hollensbe wrote:
>> On Monday 29 September 2008 12:31:55 Michael Glaesemann wrote:
>>>
>>> Any help or further debugging pointers would be much appreciated.
>>
>> It looks like the 'pg' library did not install correctly, or for some
>> reason it can't dlopen libpq. I just throw bugs in their direction
>> occasionally, and do not work on the project itself.
>>
>> http://rubyforge.org/projects/ruby-pg should be the place to report  
>> such
>> things; be certain to give them the output of ldd and a test case  
>> that
>> doesn't involve DBI, as it seems DBI's shoddy error reporting is  
>> the only
>> issue here (which has a long, outstanding ticket that I should get  
>> around
>> to resolving).
>
> Eh, to elaborate:
>
> ruby -rubygems -e 'require "pg"' should be enough to prove your  
> failure case.

Thanks, Erik, for the explanation and the concise test case! Off to  
more debugging and the ruby-pg folks :)

Michael Glaesemann
grzm seespotcode net




From george.moschovitis at gmail.com  Tue Sep 30 08:53:27 2008
From: george.moschovitis at gmail.com (Georgios Moschovitis)
Date: Tue, 30 Sep 2008 15:53:27 +0300
Subject: [ruby-dbi-users] Mysql timestamp/datetime problem
Message-ID: <afd80b540809300553u45517edcjd30bdd77763d9743@mail.gmail.com>

I am using dbi-0.4.0 and dbd-mysql-0.4.1 and I have the following strange
problem:

On my development machine, DBI select_all returns no rows if the table
contains a timestamp or datetime column.
If I drop the column, select_all returns rows.

On my staging server, everything seems to work.

Any idea what might be happening?

Thank you in advance,
George Moschovitis

-- 
http://nitroproject.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/ruby-dbi-users/attachments/20080930/b8fb8c33/attachment.html>

From erik at hollensbe.org  Tue Sep 30 10:45:04 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Tue, 30 Sep 2008 07:45:04 -0700
Subject: [ruby-dbi-users] Mysql timestamp/datetime problem
In-Reply-To: <afd80b540809300553u45517edcjd30bdd77763d9743@mail.gmail.com>
References: <afd80b540809300553u45517edcjd30bdd77763d9743@mail.gmail.com>
Message-ID: <200809300745.04569.erik@hollensbe.org>

On Tuesday 30 September 2008 05:53:27 Georgios Moschovitis wrote:

> On my development machine, DBI select_all returns no rows if the table
> contains a timestamp or datetime column.
> If I drop the column, select_all returns rows.
>
> On my staging server, everything seems to work.

The email you sent me mentioned that this involves the default result, in the 
working case, the default was null, in the failing case, the default 
was '0000-00-00 00:00:00'

The problem with the latter case is that it can't be coerced to a DateTime 
object because it's not a valid time... Honestly, I'm surprised mysql allows 
it at all. Either way, you'll need to turn type conversion off (see 
DBI.convert_types=) to get this default value to work or change it and all 
occurrences of it in your database, or use bind_coltype to treat it as string 
instead.

Hope this helps,

-Erik

From erik at hollensbe.org  Tue Sep 30 10:48:35 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Tue, 30 Sep 2008 07:48:35 -0700
Subject: [ruby-dbi-users] Proposing patch for sqlite3 driver
In-Reply-To: <m3bpy6gjel.fsf@zeus.intra.softace.dk>
References: <m3wsguu7rf.fsf@zeus.intra.softace.dk>
	<200809292242.39040.erik@hollensbe.org>
	<m3bpy6gjel.fsf@zeus.intra.softace.dk>
Message-ID: <200809300748.35952.erik@hollensbe.org>

On Tuesday 30 September 2008 00:52:34 Jarl Friis wrote:
> I hope you also get the opportunity to have a look at the DBD:Pg
> patch. If you want trigger that defect you need to configure the
> postgres server to use non-US datestyle, in a european installation
> /var/lib/pgsql/data/postgresql.conf will typically contain "datestyle
> = 'iso, dmy'", but in a US installation I guess this will be
> "datestyle = 'iso, mdy'". This is why the PG tests fails on my
> european configuration.

The problem with this is that while postgres's date format is variable, the 
parsing routes we use with DateTime are not. A smarter solution needs to be 
used, as a "strptime until it doesn't break" solution will likely cause more 
problems than it resolves.

I'll look at this on the weekend. Thanks for the reminder.

-Erik

From erik at hollensbe.org  Tue Sep 30 11:03:20 2008
From: erik at hollensbe.org (Erik Hollensbe)
Date: Tue, 30 Sep 2008 08:03:20 -0700
Subject: [ruby-dbi-users] Fwd: Re:  Mysql timestamp/datetime problem
Message-ID: <200809300803.20225.erik@hollensbe.org>

Wanted to make sure this gets to the list. This is a bug. :) The solution (for 
now) is to insert it as a literal string.

-Erik
-------------- next part --------------
An embedded message was scrubbed...
From: "Georgios Moschovitis" <george.moschovitis at gmail.com>
Subject: Re: [ruby-dbi-users] Mysql timestamp/datetime problem
Date: Tue, 30 Sep 2008 17:49:55 +0300
Size: 6043
URL: <http://rubyforge.org/pipermail/ruby-dbi-users/attachments/20080930/09a05f13/attachment.mht>