Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-11-18:

[7:15] <ondras> Wes-: please let me know when you have a sec
[11:58] <Wes-> ondras: Good morning!
[11:58] <ondras> Wes-: morning; its 13.29 here :)
[11:58] <ondras> Wes-: first, what binary proposal(s) do you have implemented in gpsee ?
[12:00] <Wes-> ondras: 0729 here. Binary/B implemented in GPSEE, plus a few extras I needed for a good C FFI
[12:00] <ondras> okay
[12:00] <ondras> I need to somehow re-think how is the binary module to be used from the c/c++ side
[12:00] <Wes-> What do you mean?
[12:01] <ondras> so, say I have a native module
[12:01] <ondras> which needs to provide binary data
[12:01] <Wes-> In GPSEE I have a concept called a "byte thing", and a C helper to help instanciate byte things
[12:02] <ondras> yeah, the same in v8cgi
[12:02] <ondras> lets stay with your terminology
[12:02] <Wes-> Byte things all have the same basic underlying C-side storage, a pointer a length
[12:02] <ondras> so, my native module needs to include a bytething.h
[12:02] <Wes-> I can cast between them in JS if I need to by using another ByteThing's constructor without "new"
[12:03] <ondras> say my c code wants to provide a char* as a ByteString instance
[12:03] <Wes-> Although "cast" is not 100% correct, because I might copy the data (based on mutability indicators, and copy-on-write semantics, but that's a whole other story)
[12:04] <ondras> what do I need to include/call in order to accomplish that?
[12:04] <Wes-> ondras: What I do in this case is have the C code return a byte thing, and then I cast in JS
[12:04] <Wes-> The casting helps both the syntax AND the .so library loading
[12:05] <Wes-> And also the garbage-collector visibility of the module, but I think I am the only one that supports module un-loading
[12:05] <Wes-> So, for example, I might have
[12:05] <Wes-> var s = new (require("SQLite").errorString)(123);
[12:06] <Wes-> var bs = require("binary").ByteString(s)
[12:06] <Wes-> ...now I don't have to manage loading the binary module and reaching across DSOs for symbols
[12:07] <ondras> hmh
[12:07] <Wes-> Which can be extremely painful if you have a flexible infrastructure that allows users to provide arbitrary modules in .so files
[12:07] <ondras> 13:34 < ondras> say my c code wants to provide a char* as a ByteString instance
[12:07] <Wes-> ondras: If you don't want the intermediate step and were using GPSEE, I would recommend writing a blended module
[12:07] <Wes-> So the last step - going from the byte thing to the ByteString is still done in JS
[12:08] <ondras> hmh
[12:08] <ondras> I see
[12:08] <ondras> so there is no way to instantialize ByteString from C ?
[12:08] <Wes-> Well, I mean, technically it's possible, but it's ugly as sin
[12:09] <Wes-> You have to first expose require() to C to make sure it can load the right .so file
[12:09] <ondras> that is the point I am slowly getting to: my require() is already implemented by means of C function
[12:09] <Wes-> Then you have to expose an API which will give you a dlopen() handle on the right .so
[12:09] <ondras> so why not use it to get to ByteString from C
[12:10] <Wes-> Then you find the binary constructor with dlsym, create the same scaffold the JS engine would normally use to call it, and call it
[12:10] * ondras is afraid that we are talking somewhat orthogonally now
[12:11] <ondras> the problem is that i have no experience with SM embedding
[12:11] <ondras> so my terminology/approach might seem completely chaotic here :/
[12:11] <ondras> but generally, a JS function has its C counterpart, okay?
[12:12] <Wes-> ondras: I don't instanciate ByteStrings from C because doing so is a) complex b) fragile c) creates garbage collection hazard (if the binary module wants to unload) d) only causes me to execute a few extra JS instructors
[12:12] <Wes-> ondras: That's not quite true, especially in the case where you're invoking constructors
[12:12] <Wes-> But even if is, when your functions are in .so files, you either have to jump through a LOT of hoops, or link them against each other at compile time
[12:13] <ondras> the code I hope I will be able to use
[12:13] <Wes-> cross-linking is a poor architectural choice here IMHO because modules need to be completely modular
[12:13] <ondras> in order to expose binary data as ByteString instance
[12:13] <ondras> can look like:
[12:13] <ondras> myv8app->require("binary")->Get("ByteString")->NewInstance(char * wrapped as byte thing)
[12:14] <ondras> does this make sense?
[12:14] <Wes-> You know, if you write your code carefully, you might be able to just steal ByteString's prototype
[12:14] <ondras> (c++ code)
[12:14] <Wes-> ondras: That would be about 3 pages of code in spidermonkey/gpsee, and still not solve the garbage collector problem
[12:15] <ondras> I believe this creates no GC hazard in v8
[12:15] <Wes-> Although - Get("ByteString") must be approximately equivalent to a call into JS
[12:15] <Wes-> meaning you're not really avoiding going to the interpreter
[12:16] <ondras> Get is a v8 api call
[12:16] <Wes-> And - of course you know this - you're poking at a specific solution whereas I have a general technique because I have many, many byte thing types that can interchange methods etc
[12:16] <Wes-> ondras: Get might be an API call, but it has to use interpreter context to resolve the symbol
[12:16] <ondras> yes, it does
[12:17] <ondras> I am still not sure whether SM/V8 are so different, or if I just cannot properly explain myself :/
[12:18] <Wes-> So, the cost of doing that with/without a blended module are *probably* on the same order
[12:18] <Wes-> ondras: Well, first off, there is no "Get" in SM, I have to eval if I want to do that
[12:18] <ondras> ah
[12:18] <Wes-> And JS Objects do not reflect as C objects
[12:18] <Wes-> So I have to write all the scaffolding back and forth by hand
[12:19] <Wes-> And then there is that issue that you don't care about the require("binary") symbol's GC lifetime because you don't unload unused modules, so you don't have to worry about telling the interpreter about it
[12:19] <Wes-> And invoking JS contructors from C is also not an API call
[12:20] <Wes-> So there are fundamental differences I guess that drive two different approaches to merging the two types :)
[12:20] <ondras> ookay
[12:20] <ondras> I thought that gpsee supports *explicit* unloading
[12:20] <Wes-> The very last statement you typed -- I would implement as JS until it was profiled as a hot-spot
[12:20] <ondras> but if you do that automatically..
[12:20] <Wes-> ondras: No, explicit unloading is dangerous. Unloading is done based on GC info
[12:21] <Wes-> If the module's exports are not reachable, and the module indicates that it has no internal state, it will be unloaded
[12:21] <Wes-> This makes it possible to upgrade modules without restarting the engine in some cases
[12:22] <Wes-> We might also support unloading JS modules; might because I'm not sure the GC will ever actually find them unreachable
[12:22] <Wes-> Ah, right, I have them pinned these days because there is no mechanism for the JS module to say it has no internal state
[12:23] * Wes- runs for his 2nd cup of coffee
[12:23] <ondras> well, thanks for consultation, I will now try to implement some native module which actually uses binary :)
[12:25] <Wes-> ondras: You're welcome!
[12:25] * Wes- goes back to cursing internet explorer
[12:26] <ondras> :))
[12:34] <Wes-> I have an error on line 426 -- of a file with only ~50 lines in it
[12:34] * Wes- hates IE
[12:51] <ondras> Wes-: IE starting from version 8 offers more reliable debugging tools than previous versions
[12:51] <ondras> I can highly recommend starting with those
[13:06] <ondras> Wes-: http://pastebin.com/uGakYLUv
[13:06] <ondras> Wes-: this seems to be the most intuitive way for a native module to expose binary data, in v8cgi
[13:41] <Wes-> ondras: Yeah - I should head to work so I can use a windows box. WINE is pretty much limited to IE6
[13:42] <Wes-> ondras: That does look nice and straightforward. One thing to keep in the back of your head; some day you may want to expose structs as objects, etc
[13:42] <ondras> virtualbox seems to be a solution for ie6 as well :)
[13:43] <Wes-> Yeah. Although I must admit, just clicking on the blue e on my mac desktop and having IE6 pop up is rather hilarious
[13:43] <ondras> :P
[13:43] <Wes-> The emulation is so perfect even the fonts suck!
[19:31] <Wes> kriszyp: I got 404 handling working on both firefox and ie8; algorithm posted to newsgroup
[19:47] <Wes> deanlandolt: you around today?
[22:01] <Dantman> Ugh... json-template... yet again

 

 

Logs by date :