[racket] frequent crashes on Windows 7 64-bit

The FFI documentation currently defers to Inside for information on
allocation and interior pointers. It should all be rewritten, but maybe
not today.

At Sat, 2 Nov 2013 08:57:28 -0400, Sam Tobin-Hochstadt wrote:
> Should the documentation mention this at the ffi level as well as in the
> Inside manual?
> 
> Sam
> On Nov 2, 2013 12:49 AM, "Matthew Flatt" <mflatt at cs.utah.edu> wrote:
> 
> > I tracked down the problem in the SQLite binding and Racket's GC.
> >
> > It turns out that an "interior" pointer retains an enclosing allocated
> > object only when the pointer refers to an even address. A pointer that
> > refers to an odd address is always treated by the GC as a fixnum (and
> > it's not practical to change the GC to support odd-valued pointers into
> > an allocated object). I've changed the documentation of
> > scheme_malloc_allow_interior() to explain the constraint.
> >
> > Meanwhile, statement preparation in the SQLite API can return an
> > odd-valued pointer into the middle of a query string (which had been
> > allocated as 'atomic-interior). If a GC occurred at just the wrong
> > time, then statement preparation could go wrong --- more so on Windows
> > than other platforms, since the GC releases pages more eagerly on
> > Windows. I've adjusted the SQLite binding to work within the newly
> > documented constraint on interior pointers.
> >
> > I didn't find any other code that could end up with odd-valued
> > pointers, but FFI users beware.
> >
> > At Wed, 30 Oct 2013 13:47:19 -0400, Ryan Culpepper wrote:
> > > I don't have an answer about the crash, but Matthew has narrowed it down
> > > to the vicinity of statement preparation, so I can make a recommendation
> > > about how to avoid it: Use a (constant) parameterized statement instead
> > > of string-append. For example, replace the main function with this:
> > >
> > > (define (main n)
> > >    (call-with-transaction
> > >     (current-db)
> > >     (lambda ()
> > >       (for ([i (expt 10 n)])
> > >         (define x_i (x i))
> > >         (query-exec
> > >          (current-db)
> > >          "INSERT INTO sines (id, x, sine_x) VALUES (?,?,?)"
> > >          i x_i (sin x_i))))))
> > >
> > > With that change, I'm able to run (main 6) in about 30 seconds and (main
> > > 7) in about 5 minutes on a 2GB, 1 processor Windows VM running within
> > Linux.
> > >
> > > The db connection caches prepared statements inside of transactions, so
> > > by using a constant parameterized statement, you save parsing time. By
> > > my measurements, this version takes less than half the time of
> > > Matthias's. (In addition to other benefits like security from SQL
> > > injection.)
> > >
> > > Ryan
> > >
> > >
> > > On 10/29/2013 06:19 PM, Andrews, Kyle (KC) wrote:
> > > > I just tried running your code 3 times on `(main 7)' and twice on
> > `(main 6)'
> > > and it has crashed each time. But I was worried that somehow I was
> > running out
> > > of memory (even though I have 16 Gb available), so  I saved the database
> > to
> > > file using:
> > > >
> > > > (define current-db (make-parameter (sqlite3-connect #:database
> > "buggy.db"
> > > #:mode 'create)))
> > > >
> > > > [Note that if you want to run the script again after this, you have to
> > > comment out the next line and remove the #:mode 'create keyword]
> > > >
> > > > DrRacket still crashes with (main 6) and (main 7).  Since (main 7)
> > should
> > > create a table with 10^7 = 10 million rows and 3 columns, I don't see how
> > > SQLite could be running out of space.  SQLite documentation [1] says it
> > should
> > > be able to support at least 10^13 rows and 140 terabytes of data.
> >  Otherwise,
> > > I'm not sure what other limits I could be running into.  And my actual
> > problem
> > > is also using an on-disk database, for which I have managed to store a
> > paltry
> > > 47 mb (which is something like 1/300 of what I need to load).
> > > >
> > > > [1] http://www.sqlite.org/limits.html
> > > >
> > > > P.S. -- Happy Birthday!  (I saw the Easter egg in DrRacket today)
> > > >
> > > > -----Original Message-----
> > > > From: Matthias Felleisen [mailto:matthias at ccs.neu.edu]
> > > > Sent: Tuesday, October 29, 2013 4:51 PM
> > > > To: Andrews, Kyle (KC)
> > > > Cc: Racket mailing list (users at racket-lang.org)
> > > > Subject: Re: [racket] frequent crashes on Windows 7 64-bit
> > > >
> > > >
> > > > Once I had fixed the typo in your program, I was able to run this loop
> > > without ado:
> > > >
> > > >> (for ((i (in-range 9))) (collect-garbage) (collect-garbage) (time
> > > >> (main i)))
> > > > cpu time: 1 real time: 1 gc time: 0
> > > > cpu time: 5 real time: 6 gc time: 0
> > > > cpu time: 9 real time: 9 gc time: 0
> > > > cpu time: 77 real time: 77 gc time: 0
> > > > cpu time: 736 real time: 741 gc time: 14 cpu time: 7666 real time:
> > 7703 gc
> > > time: 393 ...
> > > >
> > > > As far as I can tell time consumption grows by one order of magnitude
> > as
> > > increase the size of the database entries by an order of magnitude. I
> > don't
> > > have the time to it for 10, 11, and 12 but is it possible that something
> > > external/size-wise hits you here?
> > > >
> > > >
> > > >
> > > > #lang racket
> > > >
> > > > (require db)
> > > >
> > > > (define current-db (make-parameter (sqlite3-connect #:database
> > 'memory)))
> > > >
> > > > (query-exec (current-db) "CREATE TABLE sines(id INTEGER, x, sine_x)")
> > > >
> > > > (define (x i) (+ (- pi) (/ i (* 1000 pi))))
> > > >
> > > > (define (main n)
> > > >    (call-with-transaction
> > > >     (current-db)
> > > >     (lambda ()
> > > >       (for ([i (expt 10 n)])
> > > >         (define x_i (x i))
> > > >         (query-exec
> > > >          (current-db)
> > > >          (string-append
> > > >           "INSERT INTO sines(id, x, sine_x) VALUES("
> > > >           (number->string i) ", "
> > > >           (number->string x_i) ", "
> > > >           (number->string (sin x_i)) ")"))))))
> > > >
> > > > (provide main)
> > > >
> > > >
> > > >
> > > > On Oct 29, 2013, at 4:32 PM, "Andrews, Kyle (KC)" <KCAndrews at dow.com>
> > wrote:
> > > >
> > > >> I've been trying to upload data into an SQLite database using Racket
> > from
> > > Windows 7 64-bit and both Racket.exe and DrRacket.exe keep crashing
> > before the
> > > upload finishes.  I've had crashes on long-running programs in DrRacket
> > on
> > > Windows 7 before, so I don't want to say for sure that it's the fault of
> > the
> > > db library.  However, I do know is that the crashes have been happening
> > > repeatedly with this project and given the amount of data I need to
> > upload, it
> > > is impractical to keep restarting Racket in the hopes that it will
> > eventually
> > > finish.  Racket is crashing with both 5.3.6 as well as the latest nightly
> > > build you have installers for:   5.90.0.9.  I have attached a sample
> > script
> > > which creates an in-memory sqlite database and populates it with a
> > trillion
> > > sines which crashes on me after a while, just like my actual script.  My
> > > Racket VM (at least in DrRacket) defaults to 2048 Mb.
> > > >>
> > > >> Below is what error information Windows gives upon the crash:
> > > >>
> > > >> Problem signature:
> > > >>    Problem Event Name: APPCRASH
> > > >>    Application Name:        Racket.exe
> > > >>    Application Version:    5.90.0.1
> > > >>    Application Timestamp:             525e4550
> > > >>    Fault Module Name:    libracket3m_9r9qx5.dll
> > > >>    Fault Module Version:                0.0.0.0
> > > >>    Fault Module Timestamp:         525e454e
> > > >>    Exception Code:             c0000005
> > > >>    Exception Offset:          00000000002b6081
> > > >>    OS Version:      6.1.7601.2.1.0.256.4
> > > >>    Locale ID:          1033
> > > >>    Additional Information 1:          9e37
> > > >>    Additional Information 2:          9e37a01f62fc211801609d8b20878795
> > > >>    Additional Information 3:          4bbb
> > > >>    Additional Information 4:          4bbb32ba4ab4514f5750e1fc710f6223
> > > >>
> > > >> Let me know if there is any other information I can provide.
> > > >> Regards,
> > > >>
> > > >> Kyle
> > > >>
> > > >> <bug.rkt>____________________
> > > >>   Racket Users list:
> > > >>   http://lists.racket-lang.org/users
> > > >
> > > >
> > > >
> > > >
> > > > ____________________
> > > >    Racket Users list:
> > > >    http://lists.racket-lang.org/users
> > > >
> > >
> > > ____________________
> > >   Racket Users list:
> > >   http://lists.racket-lang.org/users
> > ____________________
> >   Racket Users list:
> >   http://lists.racket-lang.org/users
> >