Should the documentation mention this at the ffi level as well as in the Inside manual? Sam <div class="gmail_quote">On Nov 2, 2013 12:49 AM, "Matthew Flatt" <<a href="mailto:mflatt@cs.utah.edu">mflatt@cs.utah.edu</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I tracked down the problem in the SQLite binding and Racket's GC. It turns out that an "interior" pointer retains an enclosing allocated object only when the pointer refers to an even address. A pointer that refers to an odd address is always treated by the GC as a fixnum (and it's not practical to change the GC to support odd-valued pointers into an allocated object). I've changed the documentation of scheme_malloc_allow_interior() to explain the constraint. Meanwhile, statement preparation in the SQLite API can return an odd-valued pointer into the middle of a query string (which had been allocated as 'atomic-interior). If a GC occurred at just the wrong time, then statement preparation could go wrong --- more so on Windows than other platforms, since the GC releases pages more eagerly on Windows. I've adjusted the SQLite binding to work within the newly documented constraint on interior pointers. I didn't find any other code that could end up with odd-valued pointers, but FFI users beware. At Wed, 30 Oct 2013 13:47:19 -0400, Ryan Culpepper wrote: > I don't have an answer about the crash, but Matthew has narrowed it down > to the vicinity of statement preparation, so I can make a recommendation > about how to avoid it: Use a (constant) parameterized statement instead > of string-append. For example, replace the main function with this: > > (define (main n) > (call-with-transaction > (current-db) > (lambda () > (for ([i (expt 10 n)]) > (define x_i (x i)) > (query-exec > (current-db) > "INSERT INTO sines (id, x, sine_x) VALUES (?,?,?)" > i x_i (sin x_i)))))) > > With that change, I'm able to run (main 6) in about 30 seconds and (main > 7) in about 5 minutes on a 2GB, 1 processor Windows VM running within Linux. > > The db connection caches prepared statements inside of transactions, so > by using a constant parameterized statement, you save parsing time. By > my measurements, this version takes less than half the time of > Matthias's. (In addition to other benefits like security from SQL > injection.) > > Ryan > > > On 10/29/2013 06:19 PM, Andrews, Kyle (KC) wrote: > > I just tried running your code 3 times on `(main 7)' and twice on `(main 6)' > and it has crashed each time. But I was worried that somehow I was running out > of memory (even though I have 16 Gb available), so I saved the database to > file using: > > > > (define current-db (make-parameter (sqlite3-connect #:database "buggy.db" > #:mode 'create))) > > > > [Note that if you want to run the script again after this, you have to > comment out the next line and remove the #:mode 'create keyword] > > > > DrRacket still crashes with (main 6) and (main 7). Since (main 7) should > create a table with 10^7 = 10 million rows and 3 columns, I don't see how > SQLite could be running out of space. SQLite documentation [1] says it should > be able to support at least 10^13 rows and 140 terabytes of data. Otherwise, > I'm not sure what other limits I could be running into. And my actual problem > is also using an on-disk database, for which I have managed to store a paltry > 47 mb (which is something like 1/300 of what I need to load). > > > > [1] <a href="http://www.sqlite.org/limits.html" target="_blank">http://www.sqlite.org/limits.html</a> > > > > P.S. -- Happy Birthday! (I saw the Easter egg in DrRacket today) > > > > -----Original Message----- > > From: Matthias Felleisen [mailto:<a href="mailto:matthias@ccs.neu.edu">matthias@ccs.neu.edu</a>] > > Sent: Tuesday, October 29, 2013 4:51 PM > > To: Andrews, Kyle (KC) > > Cc: Racket mailing list (<a href="mailto:users@racket-lang.org">users@racket-lang.org</a>) > > Subject: Re: [racket] frequent crashes on Windows 7 64-bit > > > > > > Once I had fixed the typo in your program, I was able to run this loop > without ado: > > > >> (for ((i (in-range 9))) (collect-garbage) (collect-garbage) (time > >> (main i))) > > cpu time: 1 real time: 1 gc time: 0 > > cpu time: 5 real time: 6 gc time: 0 > > cpu time: 9 real time: 9 gc time: 0 > > cpu time: 77 real time: 77 gc time: 0 > > cpu time: 736 real time: 741 gc time: 14 cpu time: 7666 real time: 7703 gc > time: 393 ... > > > > As far as I can tell time consumption grows by one order of magnitude as > increase the size of the database entries by an order of magnitude. I don't > have the time to it for 10, 11, and 12 but is it possible that something > external/size-wise hits you here? > > > > > > > > #lang racket > > > > (require db) > > > > (define current-db (make-parameter (sqlite3-connect #:database 'memory))) > > > > (query-exec (current-db) "CREATE TABLE sines(id INTEGER, x, sine_x)") > > > > (define (x i) (+ (- pi) (/ i (* 1000 pi)))) > > > > (define (main n) > > (call-with-transaction > > (current-db) > > (lambda () > > (for ([i (expt 10 n)]) > > (define x_i (x i)) > > (query-exec > > (current-db) > > (string-append > > "INSERT INTO sines(id, x, sine_x) VALUES(" > > (number->string i) ", " > > (number->string x_i) ", " > > (number->string (sin x_i)) ")")))))) > > > > (provide main) > > > > > > > > On Oct 29, 2013, at 4:32 PM, "Andrews, Kyle (KC)" <<a href="mailto:KCAndrews@dow.com">KCAndrews@dow.com</a>> wrote: > > > >> I've been trying to upload data into an SQLite database using Racket from > Windows 7 64-bit and both Racket.exe and DrRacket.exe keep crashing before the > upload finishes. I've had crashes on long-running programs in DrRacket on > Windows 7 before, so I don't want to say for sure that it's the fault of the > db library. However, I do know is that the crashes have been happening > repeatedly with this project and given the amount of data I need to upload, it > is impractical to keep restarting Racket in the hopes that it will eventually > finish. Racket is crashing with both 5.3.6 as well as the latest nightly > build you have installers for: 5.90.0.9. I have attached a sample script > which creates an in-memory sqlite database and populates it with a trillion > sines which crashes on me after a while, just like my actual script. My > Racket VM (at least in DrRacket) defaults to 2048 Mb. > >> > >> Below is what error information Windows gives upon the crash: > >> > >> Problem signature: > >> Problem Event Name: APPCRASH > >> Application Name: Racket.exe > >> Application Version: 5.90.0.1 > >> Application Timestamp: 525e4550 > >> Fault Module Name: libracket3m_9r9qx5.dll > >> Fault Module Version: 0.0.0.0 > >> Fault Module Timestamp: 525e454e > >> Exception Code: c0000005 > >> Exception Offset: 00000000002b6081 > >> OS Version: 6.1.7601.2.1.0.256.4 > >> Locale ID: 1033 > >> Additional Information 1: 9e37 > >> Additional Information 2: 9e37a01f62fc211801609d8b20878795 > >> Additional Information 3: 4bbb > >> Additional Information 4: 4bbb32ba4ab4514f5750e1fc710f6223 > >> > >> Let me know if there is any other information I can provide. > >> Regards, > >> > >> Kyle > >> > >> <bug.rkt>____________________ > >> Racket Users list: > >> <a href="http://lists.racket-lang.org/users" target="_blank">http://lists.racket-lang.org/users</a> > > > > > > > > > > ____________________ > > Racket Users list: > > <a href="http://lists.racket-lang.org/users" target="_blank">http://lists.racket-lang.org/users</a> > > > > ____________________ > Racket Users list: > <a href="http://lists.racket-lang.org/users" target="_blank">http://lists.racket-lang.org/users</a> ____________________ Racket Users list: <a href="http://lists.racket-lang.org/users" target="_blank">http://lists.racket-lang.org/users</a> </blockquote></div>