Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-04-05:

[8:42] * Dantman wishes html5 would have standardized on a json based database instead of shoving a hideous sql style database into the browser.
[8:58] <ondras> http://teddziuba.com/2010/03/i-cant-wait-for-nosql-to-die.html
[8:58] <ondras> .)
[9:32] <Dantman> Bah, that's a fairly biased post that focuses completely on hype and ignores the actual pros of other database models. Besides, it's not really relevant. I doubt anyone cares about "scalability", or needs a DBA for their web browser.
[9:35] <Dantman> Browsers have very good JSON support and it ties very well to js used in them, so I'm disappointed there is no good simple way to store fair amounts of JSON data in a queryable manor. If you want something like that you have to completely break from JSON and start writing piles of code to handle databases and sql which are overboard for something inside the browser.
[9:38] <Dantman> I'm writing a tool to watch the rc of a few wiki and let you look at the new changes to pages on the wiki in unified diffs.
[9:40] <Dantman> I fetch the wiki information from the wiki in json format, I fetch the rc data from the wiki in json format. I have rc data which may have some extra log specific data I need that's not relevant to all types of rc data.
[9:42] <Dantman> That log data is completely specific to the rc item, so an extra isolated table and a join is completely unnecessary. However because of it being optional, and my need for minor querying and moderate ammounts of data, I'm going to need to setup a multi table database with a bunch of SQL.
[9:42] <Dantman> With the structure of the data I'm working with, a simple JSON based database WOULD be better.
[13:05] <Dantman> O_o as far as I can tell, html5 openDatabase's version argument and changeVersion is completely useless.
[14:40] <Wes-> Are there any programs out there which exercise narwhal-lib?
[15:24] <dmachi> Wes-: I was tryinging to laod narwhal-lib as a module in titanium yesterday, but haven't been able to figure it out yet
[15:26] <kriskowal> dmachi let me know, or fork, if you find anything that needs changing.
[15:26] <kriskowal> i wasn't aware titanium was supporting commonjs modules, although tom and i have mentioned the idea to them
[17:17] <Wes-> kriskowal: FWIW I am shopping for an app that can exercise narwhal-lib so that I can stamp "works with GPSEE" -- or possibly even distribute with GPSEE
[17:17] <kriskowal> well, tusk would be the biggest win, but it depends on a couple non-commonjs components
[17:18] <kriskowal> particularly require("narwhal/http-engine").read(url)
[17:18] <kriskowal> there's a tusk package and a zipjs package
[17:28] <ashb> kriskowal: only if you agree with the premise of how tusk works
[17:28] <ashb> i'm still not convinced by its complexity
[17:28] <ashb> or apparent complexity
[17:29] <ashb> (not looked in a fair while tho)
[17:50] <kriskowal> ashb tusk isn't complicated.
[17:51] <kriskowal> it downloads packages and puts them in a packages directory
[17:51] <kriskowal> it reads the dependency lists and makes sure all of the dependencies are downloaded
[17:51] <kriskowal> that's really the entirety of what it does
[17:51] <kriskowal> independently, the "packages" module scans the package hierarchy and assembles a require.paths based on a topological sort of the dependencies
[17:52] <kriskowal> and then the module loader. they're all orthogonal
[17:52] <kriskowal> but cohesive
[21:09] <Wes-> kriskowal: require("narwhal/http-engine").read(url) -- does that just return the contents of the URL? If so, should be able to implement cheaply
[21:09] <Wes-> (returning as a String I suppose)
[21:09] <kriskowal> yeh
[21:10] <kriskowal> it should accept {"charset": "UTF-8"} as an arg, ideally
[21:10] <kriskowal> or "b" for the mode and grab a buffer or a bytestring
[21:10] <kriskowal> but since we haven't defined that?all experimental
[21:11] <konobi> .read(url, [headers])
[21:11] <kriskowal> yeh, .read(url, {"headers": []}) would be consistent
[21:11] <kriskowal> .read is meant to mirror narwhal/fs's read(path, ?mode/options)
[21:11] <konobi> since you'll also want accept headers, auth, cookies, varies, cache control, etc.
[21:12] <kriskowal> i haven't begun to even think of proposing a standard
[21:12] <kriskowal> there are too many variables
[21:12] <konobi> LWP++
[21:12] <kriskowal> that means light-weight-processes to me; i'm guessing that's not what you meant
[21:12] <konobi> libwwwperl
[21:13] <kriskowal> ah
[21:13] <konobi> the API has really really stood the test of time
[21:14] <kriskowal> yeh, we need implementations at this point
[21:14] <kriskowal> we can't begin to dream of converging on that yet
[21:14] <kriskowal> as evidenced by our not so simple attempt to "just adopt xhr" as one http client option
[21:16] <konobi> http://search.cpan.org/~gaas/libwww-perl-5.834/lib/LWP/UserAgent.pm
[21:16] <kriskowal> Wes- just realized it might be more complex than just http.read?
[21:17] <kriskowal> http.read is implemented on require("http-engine").open(url, mode, options), which returns a readable IO stream, like IO/A basically
[21:18] <Wes-> kriskowal: *hmm*
[21:18] <kriskowal> pardon, "narwhal/http-engine", to keep it out of the commonjs namespace; just moved
[21:19] <kriskowal> narwhal/io-boot has helpers for that.
[21:19] <Wes-> kriskowal: I have a very basic implementation of Node's TCP module. NickG is also writing me a curl module. But neither of those are really IO-streamy
[21:19] <kriskowal> well, you could override narwhal/http instead of providing narwhal/http-engine, and just provide the read method.
[21:19] <Wes-> kriskowal: *idea* narwhal-lib/* for stuff in narwhal-lib/*, platform/* for bare-to-the-metal stuff?
[21:20] <Wes-> *nod* - what does read do? Return a line or a complete file?
[21:20] <kriskowal> complete file
[21:20] <Wes-> okay, that's easy to fudge with either the curl module or even cay("wget")
[21:21] <kriskowal> the idea was like fs.read(path) returns the complete content of a file by calling fs.open(path, "r").read() which in turn does the buffer accumulation until eof thing
[21:21] <Wes-> (cay ~ consume-and-yield pipeline)
[21:21] <Wes-> That makes sense
[21:21] <Wes-> It's frustrating that something like node's TCP module can't be bent into that because it is totally async
[21:21] <kriskowal> it's alright
[21:22] <kriskowal> different goals
[21:22] <Wes-> There is just isn't a good way to yield and resume inside the JS function
[21:22] <Wes-> yeah
[21:22] <kriskowal> i've been struggling with a node-based solution.
[21:22] <Wes-> Although, Neil Mix's co-routine trick might give us that if we tried hard enough. Not ES5 though.
[21:22] <kriskowal> i'm making progress
[21:23] <kriskowal> progress which is largely facilitated by node's awesome focus on async
[21:23] <Wes-> Node's TCP paradigm is really excellent for servers I have to say. We are building something ATM that I would have built in C with a bind/listen/accept/fork daemon
[21:23] <kriskowal> i fork, so the parent does a blocking wait on the child, and the child does nonblocking io with the subprocess to consume and accumulate its IO
[21:24] <Wes-> Oh, that's entertaining
[21:24] <Aristid> Wes-: i somehow get the feeling that javascript is the wrong language for node
[21:24] <kriskowal> it's a bastardization on the intent, but it'll prolly work
[21:24] <Aristid> no i'm NOT advocating c++ there
[21:24] <Wes-> How does node handle the async events when you fork? A complete copy of the reactor?
[21:25] <Wes-> Aristid: Could be thing worse than JS. I think the real lesson is that non-async is hard to implement on top of an async engine :)
[21:26] <Aristid> Wes-: oh sure, so the thing is that the language has to help you write async fluently :)
[21:27] <Wes-> I wrote a socket layer in JS that basically implemented onReadable / onWriteable events on a poll function that's built on a wide select. Sort of like thttpd's select loop. Moving that into a reactor with a richer event listener setup was pretty easy.
[21:27] <Wes-> One thing I do need to implement is a hook in GPSEE that lets me run an arbitrary function after program module completes
[21:28] <Wes-> that arbitrary function could be an arbitrary JS reactor
[21:28] <Wes-> Of course I want to move to libev at some point in the future as well
[21:28] <kriskowal> Wes- we've got that in narwhal, btw.
[21:28] <kriskowal> jetpack copied the pattern too?
[21:28] <kriskowal> i have a "event-loop-hook" module with a "when" and "emit" method
[21:29] <kriskowal> after the "main" module completes, i require the event loop hook module and emit the event, effectively
[21:29] <kriskowal> all observers get called in order
[21:29] <kriskowal> usually just one to start the event loop
[21:30] <kriskowal> we've got "event-loop", "event-loop-setup", "event-loop-engine", and "event-loop-hook"
[21:30] <kriskowal> in my "future" branch, that is.
[21:30] <kriskowal> "event-loop-engine" is expected to set up the event-loop-hook if it's required.
[21:31] <kriskowal> "event-loop" is a shallow proxy for "event-loop-engine", but it asks "event-loop-setup" for an -engine implementation, so you can provide an alternate reactor if you call setup before someone requires event-loop
[21:31] <kriskowal> so far, so good
[21:33] <Wes-> kriskowal: So, when you require your event module, it just automagically kicks off the reactor entry point at the bottom of your program module?
[21:33] <Wes-> Aristid: what's your bugmail?
[21:33] <Wes-> Aristid: for bugzilla
[21:35] <Aristid> Wes-: need to check
[21:35] <Aristid> Wes-: it's aristid.breitkreuz@gmx.de
[21:36] <kriskowal> Wes- yes
[21:36] <kriskowal> i think requiring explicit event loop reactor entry is a fail
[21:36] <Wes-> kriskowal: I DO require explicit event loop reactory entry ATM, and it IS fail

 

 

Logs by date :