[plt-dev] pgsql package

Oh, and the implicit question in there is: what should I be doing with
the planet database? Should I be using a different version of spgsql?
Should I be repairing the database by re-encoding the strings that I
find?

Thanks,
Robby

On Tue, Jul 28, 2009 at 7:36 AM, Robby
Findler<robby at eecs.northwestern.edu> wrote:
> On Tue, Jul 28, 2009 at 2:45 AM, Ryan Culpepper<ryanc at ccs.neu.edu> wrote:
>> Robby Findler wrote:
>>>
>>> Hi all (Ryan?): I've got a question about pgsql. From what I can tell,
>>> string data is stored in the database in the latin-1 encoding
>>> (sql-data.ss line 191), but is then retrieved from the database in the
>>> utf-8 encoding (io.ss line 205). Am I getting that right?
>>>
>>> This doesn't mean much, but I changed planet's copy of io.ss to use
>>> bytes->string/latin-1 instead of bytes->string/utf-8, and I was able
>>> to avoid crashing (but the latin-1 encoding might not have any
>>> unencodable octets, so that isn't really saying too too much).
>>
>> I don't know of any reason for latin-1 to show up in string code... maybe
>> for "bytea" code. It might be a mistake from the port to 200, when string
>> and bytes split. So the change sounds good.
>>
>> Were you able to get a crash before? Or data corruption?
>
> Yes, the current planet database contains the bytes #"Fl\341vio Cruz"
> as the name of one of the planet contributors. This is not a valid
> utf-8 encoding (and it does appear to be a latin-1 encoded string).
> When it is retrieved, I get the following error:
>
> bytes->string/utf-8: string is not a well-formed UTF-8 encoding:
> #"Fl\341vio Cruz"
>
>  === context ===
> parse:DataRow
> /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/p3.ss:87:5
> /local/svn/plt/collects/scheme/promise.ss:94:4: loop1
> /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:71:6:
> recv method in base%
> /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:469:6:
> query1:data-loop method in ...rivate/connection.ss:371:4
> /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:402:10:
> loop
> /home/wwwplanet/.plt-scheme/planet/300/3.99.0.25/cache/schematics/spgsql.plt/2/3/private/connection.ss:850:6:
> -map method in ...rivate/connection.ss:759:4
> get-all-users
> /local/svn/plt/collects/scheme/private/misc.ss:68:7
>
>> If you send me some code to produce it, I'll add it to the test suite.
>
> Unfortunately, I only have a database that has what appears to be bad
> data in it; I don't know how to make it happen with an empty database,
> but I guess that if you could get the line I listed to run, you'd make
> it happen.
>
>> Since the line numbers
>> you list don't correspond to recent spgsql releases, it's possible this bug
>> has already been fixed and planet is just using an old version.
>
> Oh, I was using version (2 3) of spgsql, sorry.
>
> Robby
>