[racket] frequent crashes on Windows 7 64-bit

I've switched to MySQL where so far I have not experienced any crashes (using string-append to create my queries).  However, I will definitely use '?' inside of queries from now on.

-----Original Message-----
From: Ryan Culpepper [mailto:ryanc at ccs.neu.edu] 
Sent: Wednesday, October 30, 2013 1:47 PM
To: Andrews, Kyle (KC)
Cc: Matthias Felleisen; Racket mailing list (users at racket-lang.org)
Subject: Re: [racket] frequent crashes on Windows 7 64-bit

I don't have an answer about the crash, but Matthew has narrowed it down to the vicinity of statement preparation, so I can make a recommendation about how to avoid it: Use a (constant) parameterized statement instead of string-append. For example, replace the main function with this:

(define (main n)
   (call-with-transaction
    (current-db)
    (lambda ()
      (for ([i (expt 10 n)])
        (define x_i (x i))
        (query-exec
         (current-db)
         "INSERT INTO sines (id, x, sine_x) VALUES (?,?,?)"
         i x_i (sin x_i))))))

With that change, I'm able to run (main 6) in about 30 seconds and (main
7) in about 5 minutes on a 2GB, 1 processor Windows VM running within Linux.

The db connection caches prepared statements inside of transactions, so by using a constant parameterized statement, you save parsing time. By my measurements, this version takes less than half the time of Matthias's. (In addition to other benefits like security from SQL
injection.)

Ryan


On 10/29/2013 06:19 PM, Andrews, Kyle (KC) wrote:
> I just tried running your code 3 times on `(main 7)' and twice on `(main 6)' and it has crashed each time. But I was worried that somehow I was running out of memory (even though I have 16 Gb available), so  I saved the database to file using:
>
> (define current-db (make-parameter (sqlite3-connect #:database 
> "buggy.db" #:mode 'create)))
>
> [Note that if you want to run the script again after this, you have to 
> comment out the next line and remove the #:mode 'create keyword]
>
> DrRacket still crashes with (main 6) and (main 7).  Since (main 7) should create a table with 10^7 = 10 million rows and 3 columns, I don't see how SQLite could be running out of space.  SQLite documentation [1] says it should be able to support at least 10^13 rows and 140 terabytes of data.  Otherwise, I'm not sure what other limits I could be running into.  And my actual problem is also using an on-disk database, for which I have managed to store a paltry 47 mb (which is something like 1/300 of what I need to load).
>
> [1] http://www.sqlite.org/limits.html
>
> P.S. -- Happy Birthday!  (I saw the Easter egg in DrRacket today)
>
> -----Original Message-----
> From: Matthias Felleisen [mailto:matthias at ccs.neu.edu]
> Sent: Tuesday, October 29, 2013 4:51 PM
> To: Andrews, Kyle (KC)
> Cc: Racket mailing list (users at racket-lang.org)
> Subject: Re: [racket] frequent crashes on Windows 7 64-bit
>
>
> Once I had fixed the typo in your program, I was able to run this loop without ado:
>
>> (for ((i (in-range 9))) (collect-garbage) (collect-garbage) (time 
>> (main i)))
> cpu time: 1 real time: 1 gc time: 0
> cpu time: 5 real time: 6 gc time: 0
> cpu time: 9 real time: 9 gc time: 0
> cpu time: 77 real time: 77 gc time: 0
> cpu time: 736 real time: 741 gc time: 14 cpu time: 7666 real time: 7703 gc time: 393 ...
>
> As far as I can tell time consumption grows by one order of magnitude as increase the size of the database entries by an order of magnitude. I don't have the time to it for 10, 11, and 12 but is it possible that something external/size-wise hits you here?
>
>
>
> #lang racket
>
> (require db)
>
> (define current-db (make-parameter (sqlite3-connect #:database 
> 'memory)))
>
> (query-exec (current-db) "CREATE TABLE sines(id INTEGER, x, sine_x)")
>
> (define (x i) (+ (- pi) (/ i (* 1000 pi))))
>
> (define (main n)
>    (call-with-transaction
>     (current-db)
>     (lambda ()
>       (for ([i (expt 10 n)])
>         (define x_i (x i))
>         (query-exec
>          (current-db)
>          (string-append
>           "INSERT INTO sines(id, x, sine_x) VALUES("
>           (number->string i) ", "
>           (number->string x_i) ", "
>           (number->string (sin x_i)) ")"))))))
>
> (provide main)
>
>
>
> On Oct 29, 2013, at 4:32 PM, "Andrews, Kyle (KC)" <KCAndrews at dow.com> wrote:
>
>> I've been trying to upload data into an SQLite database using Racket from Windows 7 64-bit and both Racket.exe and DrRacket.exe keep crashing before the upload finishes.  I've had crashes on long-running programs in DrRacket on Windows 7 before, so I don't want to say for sure that it's the fault of the db library.  However, I do know is that the crashes have been happening repeatedly with this project and given the amount of data I need to upload, it is impractical to keep restarting Racket in the hopes that it will eventually finish.  Racket is crashing with both 5.3.6 as well as the latest nightly build you have installers for:   5.90.0.9.  I have attached a sample script which creates an in-memory sqlite database and populates it with a trillion sines which crashes on me after a while, just like my actual script.  My Racket VM (at least in DrRacket) defaults to 2048 Mb.
>>
>> Below is what error information Windows gives upon the crash:
>>
>> Problem signature:
>>    Problem Event Name: APPCRASH
>>    Application Name:        Racket.exe
>>    Application Version:    5.90.0.1
>>    Application Timestamp:             525e4550
>>    Fault Module Name:    libracket3m_9r9qx5.dll
>>    Fault Module Version:                0.0.0.0
>>    Fault Module Timestamp:         525e454e
>>    Exception Code:             c0000005
>>    Exception Offset:          00000000002b6081
>>    OS Version:      6.1.7601.2.1.0.256.4
>>    Locale ID:          1033
>>    Additional Information 1:          9e37
>>    Additional Information 2:          9e37a01f62fc211801609d8b20878795
>>    Additional Information 3:          4bbb
>>    Additional Information 4:          4bbb32ba4ab4514f5750e1fc710f6223
>>
>> Let me know if there is any other information I can provide.
>> Regards,
>>
>> Kyle
>>
>> <bug.rkt>____________________
>>   Racket Users list:
>>   http://lists.racket-lang.org/users
>
>
>
>
> ____________________
>    Racket Users list:
>    http://lists.racket-lang.org/users
>