Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2009-12-08:

[1:02] <Wes-_> deanlandolt: Thanks! So I guess I'm not the only one who thinks undefined is a type, that's great news! :)
[2:11] <Edgar1> hello, i have sometime reading about commonjs and the goal...I would like to know in what step is in the project right now :)
[2:20] * Edgar1 no response yet :-/
[4:14] <deanlandolt> Edgar1: i believe we're in step 7 of the 12 steps: "Humbly asked Him to remove our shortcomings."
[4:33] <Wes-_> deanlandolt: Funny, I thought we were still at "admit we have a problem"
[4:45] <okito> hi anyone in?
[4:45] <okito> what is the status of this little doozy?
[4:45] <okito> http://github.com/ashb/interoperablejs/blob/master/extensions/imports/test.js
[4:48] <okito> ashb: this looks like your doing ^^ :-)
[12:19] <ashb> But really for..in is pretty sick for arrays. It will convert every
[12:19] <ashb> single index in the array to a string.
[12:19] <ashb> (in V8)
[12:19] <ashb> it'll do what?
[12:23] <ondras> for..in in arrays is plain evil
[12:23] <ondras> and a big misleading argument for all js haters out there
[12:23] <ashb> yeah but why would it do any kind of conversion?
[12:24] <ashb> does the spec really say for..in has to return string values only?
[12:25] <ondras> >>> var arr = [1,2,3];
[12:25] <ondras> >>> for (var p in arr) console.log(JSON.stringify(p))
[12:25] <ondras> "0"
[12:25] <ondras> "1"
[12:25] <ondras> "2"
[12:25] <ondras> hmhm
[12:26] <ashb> also for each..in ftw
[12:28] <ondras> so, looking at ecma-262
[12:28] <ondras> Let P be the name of the next property of obj
[12:28] <ondras> "name of the property" looks like a string value
[12:33] <ondras> and it makes some sense to me -- all object keys are strings...
[13:44] <ashb> Dantman: i *think* my api has the advtange that it is easier to write body changing middleware (such as a gzip middlewarE)
[15:25] <Wes-> ashb: body-changing middleware is extremely important. "apache 2 filters"
[15:25] <ashb> yeah
[15:26] <ashb> ESI debug filter would be another example i could see
[15:32] <Wes-> ashb: possibly also an interesting way to do i18n
[15:33] <ashb> okito: that interopablejs github reop is mine. i've not updated it in a while - i probably should re-update it
[15:35] <okito> oh ok thanks
[15:35] <okito> so I should not try to make those unit tests pass
[15:40] <ashb> not sure how out of date
[15:40] <ashb> but no not just yet
[15:40] <ashb> i'll update them today
[16:27] <ashb> JohnResi1: got some questions about Sizzle - where's the best place to discuss them
[16:33] <ashb> also wow the full dom is stupidly complex
[17:04] <ducktype> if i don't want to use java, the only option is Zest/Juice?
[17:04] <ashb> yes :)
[17:04] <ashb> its not tho
[17:05] <ashb> ducktype: there is also v8cgi
[17:05] <evilstreak> Zest/Juice is the coolest though <_<
[17:05] <ashb> and i think narwhal has a JAvaScriptCore engine, but it might not be complete, and might still need rhino to bootstrap up. not sure
[17:05] <ashb> http://gist.github.com/251804
[17:06] <ducktype> v8cgi has fastcgi so i can use it with nginx
[17:07] <ducktype> it's possible to make a bridge from fastcgi > jsgi or they are so different?
[17:07] <ashb> if you want fastcgi I can write one for Juice by tomororw
[17:08] <ducktype> oh ok so fastcgi request can be traslated into a jsgi
[17:08] <ashb> yeah
[17:08] <ducktype> ashb: lol tomorrow :)
[17:08] <ashb> i've written an old version of fastcgi bindings, so i'd just have to update them
[17:09] <ashb> (as in for not JSGI, but for a previouse web framework)
[17:09] <ducktype> because i'm thinking, that could be better if we have a webserver > fcgi-deamon > multi-language-workers
[17:10] <ashb> hmmm interesting
[17:10] <ducktype> also php is implementing something like that at the moment php-fpm
[17:10] <ducktype> php sucks and needs one worker for req at time (can be reaused)
[17:11] <ducktype> but for js/other we can use threads inside one worker or could be configurable
[17:12] <ducktype> also this will make possible to have script in any lang executed impersonating the right user
[17:12] <ducktype> security/hosting needs etc
[17:12] <ashb> so you want me to write a fcgi module then?
[17:13] <ducktype> no we need a fcgi-worker protocol..
[17:13] <ashb> could that not be fcgi itself? :)
[17:13] <ashb> fcgi in and out
[17:13] <ducktype> yes
[17:13] <ducktype> but there is one point problematic
[17:14] <ashb> fcgi make me die inside a little bit each time? ;)
[17:14] <ducktype> eheh :)
[17:15] <ducktype> the point is: we dont' want too many pipes between the man fcgi deamon and the workers
[17:15] <ducktype> but we need to know the path of the script (any lang) top be executed prior to choose or spawn a worker
[17:16] <ducktype> becouse it's very usefult to switch security context depending on the path
[17:16] <ducktype> /home/USER/public_html/app/script.lang
[17:17] <ashb> intersting idea. do you have an actualy usecase in mind here or just thinking of cool things we could do?
[17:17] <ducktype> argh i need i good diagram to ecxplain that :|
[17:18] <ducktype> i want to host php/js in the same in the company i work in
[17:18] <ducktype> but is small and crappy :)
[17:19] <ducktype> also i'm following for months the php-fpm project and i see something to be shared and that is not language dependant
[17:19] <ashb> so what would this dispatcher daemon do that the webserver couldn't?
[17:19] <ashb> just be more configurable about what scripts get run as what user etc?
[17:20] <ducktype> yes the fcgi-process-manager-deamon accept fcgi reqs from the web server and spawn/reause a process of the same language type and security context identified by the request
[17:20] <ducktype> spawn/reause a worker process
[17:21] <ducktype> so any language that support the worker process protocol (fcgi as you pointed out) but with some small new thingy or convention can be pluggend in
[17:21] <ashb> hmmmm. I like that idea
[17:21] <ducktype> cool :)
[17:22] <ducktype> you are the first one :)
[17:22] <ducktype> i'll try to put in a good diagram and woriking of a libevent based implementation of the fcgi-process-manager-deamon
[17:23] <ashb> the libevent is implementation. 'evented' is all it needs
[17:23] <ducktype> do you think boost:asio or libev are better impl?
[17:24] <ducktype> yes but the point is evented is better :)
[17:24] <ashb> libev and asio are def better than libevent, yeah
[17:27] <ducktype> the process-manager can chroot / impersonate based on path regexp or fixed user / choose the right worker binary based on the extension? / define pools of workers re-spawn , adaptively incerase or decrease workers on pools, log stdout/stderr of workers , keep statistics about pools/workers, restart single pools of workers etc...
[17:27] * Dantman wonders how much longer it'll take to finish porting NodeBox Graph
[17:30] <ducktype> so instead of having phusion for rails / php-fpm for php / zest etc for js etc etc
[17:30] <ashb> ducktype: sounds cool. want to work on it with me?
[17:30] <ducktype> we have languages that support the "fcgi worker mode"
[17:30] <ashb> +
[17:30] <ashb> hmmm random key
[17:31] <ducktype> ashb: sure i'm searching/speaking for a while to find someone on that
[17:31] <ashb> I don't have an immediate need for it
[17:32] <ducktype> having php/flusspferd/v8cgi workers will we cool
[17:32] <ashb> but i would use it if it existed
[17:32] <ducktype> me too but i think it's missing piece of the new multilang web stack
[17:33] <ducktype> json for config
[17:34] <ducktype> there is a good C++ lib to handle json/conf with includes?
[17:34] <ashb> spidermonkey does an okay job ;)
[17:34] <ashb> but no, includes would have to be an extension to it
[17:35] <ducktype> i think on using this json similar: http://www.hyperrealm.com/libconfig/
[17:35] <ashb> i've been thinking of writing a PEG based JSON parser
[17:35] <ashb> since the JSON.parse is far to strict for most uses on the web, sadly
[17:35] <ducktype> because in the deamon we don't need a full javascript engine
[17:35] <ducktype> i think
[17:36] <ducktype> "key" :(
[17:36] <ashb> true but you could just fork CoW to get it for 'free' if you are running and JS
[17:36] <ashb> ducktype: yup thats the one
[17:36] <ashb> :(
[17:36] <ducktype> what is CoW?
[17:36] <ashb> Copy on Write
[17:36] <ashb> its what makes forks so cheap
[17:37] <ashb> and keeps down actualy memory usage (virt memory will duplicat, but resident set size wont)
[17:37] <ducktype> oh no, no need i think
[17:37] <ducktype> they are persistent workers
[17:38] <ducktype> depends on how you config that
[17:38] <ducktype> so the main deamon process doen't need to share engines with workers to spawn fast
[17:38] <ashb> doesn't need to, true. it could tho
[17:39] <ashb> hmmm i think i remember something that would do this for at least ruby and python
[17:39] <Wes-> ashb: what's wrong with using eval instead of JSON.parse for your PEG?
[17:39] <ashb> you want to eval on the server?
[17:40] <ashb> with random code you might have got from a web request? :)
[17:40] <Wes-> ashb: yes, if you eval after making sure your input string matches your grammar with regexp
[17:40] <ducktype> i think in the main process we need json because for example generating statists reposnse about pools or handling json-rpc to restart pools
[17:40] <ashb> oh like that, i see.
[17:40] <Wes-> of course, you better get that grammar right!! :)
[17:41] <ducktype> Wes-: on the server we have the good one JSON.parse so no need to hack that way like in the browser
[17:41] <Wes-> ducktype: ashb just said it wasn't good enough for him
[17:43] <ashb> might be for this config case, but for a http-fetch getJSON method JSON.parse isn't good enough
[17:43] <ducktype> yes :)
[17:44] <ashb> hmmm might be worth writing an event fastcgi client
[17:44] <ashb> since libfastcgi well, sucks.
[17:44] <_ry> yes
[17:45] <ashb> even for non-evented its a pain
[17:45] <ashb> very 1995 code
[17:45] <_ry> it should be reimplemented from scratch
[17:46] <_ry> apparently there are some streaming issues with fcgi - which i don't really understand yet
[17:46] <_ry> i think it requires requests to be buffered?
[17:46] <ashb> scgi might be a better choice of protocol
[17:46] <ashb> but i guess not as mny things support it
[17:46] <_ry> yeah
[17:47] <ducktype> mmm fcgi is more powerful it can multiplex multiple channels over one tcp req
[17:47] <_ry> a libev fcgi server would be really nice
[17:47] <ducktype> and is more adopted by webservers
[17:47] <ducktype> but yes the libs sucks
[17:47] <_ry> it can also respond out of order
[17:47] <_ry> which would be great for node
[17:47] <_ry> since node can also respond out of order:)
[17:47] <ashb> 'out of order'? you mean for the different multiplexed nodes?
[17:48] <ashb> s/nodes/requests/
[17:48] <ducktype> most of web server anyway create one socket for req
[17:48] <ducktype> apache2/nginx
[17:49] <ducktype> but you can have req/stdout/stderr on the same channel
[17:49] <_ry> ashb: as opposed to persistent http which must respond ot requests in the order they came
[17:49] <ashb> ah right yes
[17:49] <ducktype> _ry: for now it's not happening
[17:50] <ducktype> but one day fcgi can do huge win over scgi
[17:50] <ducktype> i think
[17:51] <ducktype> or someone can prove apache/nginx or any othe rewbserver reuse the same conn to the fcgi deamon for multiple reqs?? :)
[17:51] <_ry> ducktype: hmm.. good question
[17:53] <ducktype> i've tryed no one multiplex :)
[17:54] <ashb> my gut tells me they dont
[17:54] <_ry> ducktype: doesn't it go over stdio?
[17:55] <ducktype> if i understand, no
[17:56] <_ry> i think in the traditional case it does
[17:56] <ashb> its all over a socket isn't it?
[17:56] <ducktype> webservers doen't spawn fcgi deamons, the connect to it by socket
[17:56] <ashb> oh if its spawned in process by the webserver i guess its stdio
[17:56] <ashb> ducktype: they can
[17:56] <ducktype> mmm never seen
[17:57] <ducktype> also mod_fcgi seems to spawn the deamon if needed but the have config in sync so the web server knows wehere to cennect locally by socket
[17:58] <ducktype> wrong?
[17:58] <ducktype> (sorry for my crippled writing :))
[18:02] <ducktype2> mmm disconnect
[18:02] <ducktype2> i use nginx and it needs a socket(tcp or unix) fastcgipass "host:port" or "unix:/var/run/poolsocket"
[18:02] <ducktype2> apache lighttpd does the same AFAIK
[18:05] <ducktype2> re
[18:07] <_ry> this is a good read for fcgi: http://www.ruby-forum.com/topic/197216
[18:19] <ducktype2> http://pastebin.com/m1fc5ddd7
[18:19] <ducktype2> a confused configi example, i hope this make sense
[18:19] <ducktype2> it's only a try
[18:22] <ducktype2> with fastcgi you can also make the upload faster because handled by webserver and then passed on file to the fcgi deamon
[18:29] <ducktype> dead?
[18:34] <ashb> hwat do wash some dishes
[18:34] <ashb> *had
[18:37] <ashb> ducktype: detecting by exteneion is probably only of marginal use
[18:37] <ashb> since i prefer most things to be extension-less
[18:40] <ducktype> with request_path i mean the filesystem path actually executed
[18:40] <ducktype> not the rewrite
[18:40] <ashb> it might also be on Host: header and/or http path
[18:40] <ducktype> sure we need to think about
[18:41] <ducktype> also the engines probably make ense as global
[18:41] <ducktype> not per pool
[18:41] <ducktype> the engines definitions
[18:41] <ashb> its a workable idea tho
[18:43] <ducktype> the point is i see it good think if we have many langs that simply have the worker implemented nothing more to do for deploy
[18:50] <ducktype> ashb: do you still like it? :)
[20:02] <Wes-> ashb: realization: CommonJS require() is sufficient to implement packages, provided
[20:03] <Wes-> - sandboxing environment provides sandbox-safe require
[20:03] <Wes-> - require is deterministic
[20:03] <Wes-> - package manifests are themselves CommonJS modules
[20:03] <Wes-> proof-of-concept implementation posted to the mailing list
[20:03] <Wes-> untested, of course
[20:03] <ashb> the last point isn't needed is it?
[20:03] <Wes-> ashb: yes, it is - you can then require() manifests
[20:04] <ashb> Wes-: ah i guess that makes it easier to sandbox
[20:04] <ashb> without having to pass in something that can otherwise read manifest files
[20:04] <Wes-> exactly! :)
[20:05] <Wes-> it also shows, IME, that require() is a sufficiently powerful small building block that we don't need to mess with it.
[20:05] <Wes-> s/E/O
[20:05] <ashb> will look in a moment.
[20:05] <ashb> trying to track down a bug in this XML library
[20:06] <ashb> crap. it doesn't distingiush between empty Value and no value
[20:06] <ashb> damn XML is such a cognative dissedence to most langauges
[20:27] <ashb> kriskowal: installing packages is irrelevant to how the package is required/loaded
[20:28] <kriskowal> not necessarily.
[20:28] <kriskowal> but, yes, it is possible to pre-acquire the transitive using packages
[20:28] <ashb> put it this way. My package loader is not going to laod modules from HTTP
[20:28] <ashb> thats the job of an installer, not the loader
[20:29] <kriskowal> sure, narwhal does the same currently. caja does not.
[20:29] <kriskowal> so, it's an orthogonal issue.
[20:29] <kriskowal> but, the using-packages loader still requires elevated authority
[20:29] <kriskowal> that it's not necessarily HTTP is a bit of a red herring.
[20:30] <ashb> i still dont get tyhe distrinctiong between the two package types
[20:30] <kriskowal> their layout is similar. the lib path is different though.
[20:30] <ashb> exactly - why the distinction?
[20:30] <kriskowal> a using-package has sovereignty over its lib path identifier namespace
[20:30] <kriskowal> it isn't an overlay
[20:31] <ashb> so its unsing-packages and the systme package?
[20:31] <ashb> using even
[20:34] <kriskowal> not sure what you mean
[20:34] <kriskowal> those are the names i'm using to distinguish them.
[20:35] <ashb> i'm not clear why they need distinguishing
[20:35] <kriskowal> system packages are the ones that a sysadmin installs on require.paths
[20:35] <ashb> well, okay, not true
[20:35] <kriskowal> using packages are the ones mapped to an internal package name space for each package in its own package descriptor
[20:35] <ashb> i dont get why the are so differentiated
[20:36] <ashb> the system package isn't a package, it is the core/system modules
[20:36] <ashb> everything else is a package
[20:36] <kriskowal> system packages do not address the issue of module name space sovereignty, but are very convenient for current system-administrator and installer practices
[20:37] <kriskowal> ah, i see. without something like tusk, i can see why you might have that vision
[20:38] <kriskowal> presently in narwhal, we have a bunch of packages that we can install into the system/core
[20:39] <kriskowal> it will make sense for some of those packages to be migrated to using-packages style, but not all.
[20:39] <ashb> stop triyng to spec exactly what you've already implemented :)
[20:39] <kriskowal> that's not really funny.
[20:40] * ashb might be cranky from palying with shitty DOM apis
[20:40] <kriskowal> you know i'm a volunteer, right?
[20:40] <kriskowal> i'm, like getting paid nothing for this.
[20:40] <ashb> neither am i
[20:40] <kriskowal> so, i've offered my service to vet ideas for this
[20:40] <kriskowal> the prototype *can* be thrown away, but it is better than nothing
[20:41] <kriskowal> and it wasn't entirely my idea
[20:41] <kriskowal> lunch. i'm done here.
[22:18] <ashb> hmmm now require.using(pkg)(id) i like
[22:26] <okito> using(pkg)(id) or using(pkg, id) ?
[22:27] <ashb> kinda would like to avoid any more top level vars
[22:27] <ashb> also require.using(pkg)(id) works better from a (english) grammar) PoV
[22:27] <ashb> using(x)...dowhat?
[22:27] <ashb> so it could be using(pkg).require(module)
[22:31] <JohnnyL> using({pkg: 'foo', id: 'bar'})
[22:31] <JohnnyL> ashb i like using(pkg).requires(module)
[22:32] <JohnnyL> or using(pkg).requiring(module)
[22:32] <ashb> requiring isn't the right tens
[22:32] <ashb> e
[22:33] <ashb> the advatange of require.using its its easier to feature test
[22:33] <ashb> if (require.using) vs if (typeof using != "undefined")
[22:33] <JohnnyL> ah ok
[22:38] * JohnnyL likes Python's from..import, except when it comes to subdirs.

 

 

Logs by date :