spontaneous mysql UTF-8 encoding

nicholas on 2009-12-31T15:38:43

Dear Lazyweb,

Has anyone seen the following - perfectly happy MySQL connection, with:

$VAR1 = [ 
          [ 
            'character_set_client',
            'latin1'
          ],
          [ 
            'character_set_connection',
            'latin1'
          ],
          [ 
            'character_set_database',
            'latin1'
          ],
          [ 
            'character_set_filesystem',
            'binary'
          ],
          [ 
            'character_set_results',
            'latin1'
          ],
          [ 
            'character_set_server',
            'latin1'
          ],
          [ 
            'character_set_system',
            'utf8'
          ],
          [ 
            'character_sets_dir',
            '/usr/share/mysql/charsets/'
          ]
        ];
(i.e. the connection is consistently latin1), and data that is Latin 1. Yet, sometimes the server will return the characters of Latin 1 encoded as UTF-8. Same query. Same table. Nothing changed.

And, yes, I know that it's the server, or at least the client libraries, because I have "instrumented" my copy of DBD::mysql:

--- dbdimp.c~   2008-04-29 00:36:17.000000000 +0100
+++ dbdimp.c    2009-12-31 14:30:58.000000000 +0000
@@ -3489,6 +3489,18 @@
     if (DBIc_TRACE_LEVEL(imp_xxh) >= 2)
       PerlIO_printf(DBILOGFP, "\t<- dbd_st_fetch, %d cols\n", num_fields);
 
+    {
+      char buffer[256];
+      sprintf(buffer, "/tmp/DBD.%d", getpid());
+      PerlIO *fh = PerlIO_open(buffer, "a");
+      if (fh) {
+       do_sv_dump(0, fh, av, 0, 32, 0, 0);
+       PerlIO_close(fh);
+      } else {
+       croak("open '%s': %d", buffer, errno);
+      }
+    }
+
     return av;
   }
   else
@@ -3601,6 +3613,17 @@
         (void) SvOK_off(sv);  /*  Field is NULL, return undef  */
     }
 
+    {
+      char buffer[256];
+      sprintf(buffer, "/tmp/DBD.%d", getpid());
+      PerlIO *fh = PerlIO_open(buffer, "a");
+      if (fh) {
+       do_sv_dump(0, fh, av, 0, 32, 0, 0);
+       PerlIO_close(fh);
+      } else {
+       croak("open '%s': %d", buffer, errno);
+      }
+    }
     if (DBIc_TRACE_LEVEL(imp_xxh) >= 2)
       PerlIO_printf(DBILOGFP, "\t<- dbd_st_fetch, %d cols\n", num_fields);
     return av;

"mysql_st_internal_execute MYSQL_VERSION_ID 50134", DBD-mysql-4.007, DBI-1.609, perl 5.8.8, x86_64 GNU/Linux


SET NAMES

mpeters on 2010-01-04T22:30:05

Could any of your code be calling "SET NAMES uft8" which changes that connection's encoding.