Mochabot log - CommonJS IRC channel: #commonjs on irc.freenode.net

2010-01-13:

[0:00] <jmerlin> anyone look into worker threads vs procs and sharing workers (like objects that are shared)?
[0:01] <ashb> most js engines dont cope very well wit hsharing objects between threads
[0:01] <jmerlin> well you wouldn't do it in JS, you'd have C++ through accessors manage the sharing
[0:01] <jmerlin> or any other parent language that supports it
[0:02] <ashb> yeah - most work on threads to my knowledge has been shared nothing/string passing
[0:05] <jmerlin> yeah i see that but i can imagine quite common scenarios in programming where the share nothing worker model would be inconvenient and locking would need to take place on objects that are being processed so as to two workers don't get tasked with things that feed back data that would need to modify the same property, with one being a much shorter request coming after a longer request, causing race conditions that JS can't really handle, those objects shoul
[0:06] <jmerlin> anyway, going home, l8r
[0:07] <ashb> share nothing doesn't need locking
[0:11] <ashb> oh hes gone
[0:13] <Dantman> Bah... screw WYSIWYG editors... I'm rolling my own inline contentEditable library.
[0:14] <ashb> WYSIWYG in general sucks
[0:16] <isaacs> yes, the best wysiwyg editors are awful. and anything awful, ported to a web browser, is like awful^2
[0:19] <ashb> man i can't belive i lost the entire day to deb packages
[0:21] <Dantman> ashb, ;) At least they didn't take your debit card.
[0:21] <ashb> https://launchpad.net/~ashb/+archive/ppa
[0:21] <ashb> still - managed to get it workign in the end
[0:30] <Dantman> This could make for a good candidate of another chunk of the system we could export as an open-source library.
[0:31] <ashb> 'this' being your editor comp?
[0:31] <Dantman> mhmm
[0:32] <Dantman> Library WYSIWYG without any UI, which can be applied to any node in the page.
[0:37] <cpojer> Dantman: most editors are *crap*
[0:37] <cpojer> in terms of code
[0:37] <cpojer> but they work
[0:37] <cpojer> if I sell something to a client I always use tinymce
[0:37] <cpojer> even though it is fat, ugly, bloated and half-broken
[0:37] <Dantman> Unfortunately they don't work in the way I want.
[0:37] <cpojer> and even though it uses sizzle
[0:38] <Dantman> Ah... out of time to work on the library. time to deal with getting my debit card back.
[0:50] <kriskowal> from #node.js, isaacs there's nothing in the packages/1.0 that implies automatic installation
[0:50] <isaacs> kriskowal: that's what i got from the cross-package reference stuff ni http://wiki.commonjs.org/wiki/Packages-Background
[0:51] <kriskowal> that's not in 1.0
[0:51] <kriskowal> that's background for potential additions in 3.0, and does not imply that it occurs at run time. it's done at install time.
[0:52] <kriskowal> also, the layout is important; and being able to override the default layout is also important.
[0:52] <isaacs> hm, i suppose it does say "possible"
[0:53] <isaacs> i agree that layout is important.
[0:53] <kriskowal> and certainly necessary in a web browser, but that's down the road.
[0:53] <isaacs> but i think it's over-specified, and will require a lot of restructuring in some existing packages.
[0:53] <kriskowal> ihab was addressing a scenario where using-packages would be compiled by caja
[0:54] <kriskowal> a lot of existing packages are definitely going to have to update their package.jsons, and tusk will have to change to accommodate the schema changes.
[0:54] <isaacs> so, here's kind of the direction i've been heading with npm...
[0:54] <kriskowal> which is inconvenient, but whatever. the choices were mostly good. "lib" is probably the only necessary layout specification.
[0:54] <kriskowal> oh, you're writing a package manager?
[0:54] <ashb> was the other chat about the need for package.json at all, or some of hte fields?
[0:55] <ashb> kriskowal: btw, i'm not writing a package manager, more of a cpan-searching tool
[0:55] <isaacs> package.json specifies an entry point module. npm hooks it up so that you can do require("foo") and get the entry point that the foo-package says you should get.
[0:55] <isaacs> kriskowal: a very hands-off manager, yeah
[0:55] <isaacs> like the kind that lets you work from home whenever you want as long as you hit your dates.
[0:55] <kriskowal> ashb, ah
[0:55] <ashb> kriskowal: http://packag.es - nothing at all there no
[0:56] <kriskowal> yeah, seen the site.
[0:56] <isaacs> as i've said before, i think that it's actually a good thing to have a few different package managers (and a few web platforms and a few everythings) at this stage.
[0:56] <kriskowal> absolutely. wouldn't be much need for a spec if there were just one.
[0:57] <isaacs> i don't mind parking my bikes in your shed.
[0:57] <ashb> my plan is basically - slupr in every package i can find, render its docs and provide searching
[0:57] <isaacs> kriskowal: this might be a stupid question...
[0:57] <kriskowal> isaacs, just want to make sure you participate in the 1.1 discussion
[0:58] <isaacs> so, you have { lib : "lib/" }, right?
[0:58] <isaacs> how do i actually USE that program sitting at lib/foo.js?
[0:58] <kriskowal> that's what is presumed if not explicated
[0:58] <isaacs> require("foo")?
[0:58] <kriskowal> with system-packages, yes.
[0:58] <kriskowal> that prolly ought to be clarified.
[0:58] <kriskowal> if it's not in there.
[0:59] <cpojer> btw. we have recently released the forge which is our official plugins repository ( http://mootools.net/forge ) and it is open source ( http://github.com/Guille/PluginsKit ) - it may be useful and would probably only require little changes to adopt the commonjs package system
[0:59] <cpojer> just fyi
[0:59] <cpojer> :D
[0:59] <kriskowal> very cool
[0:59] <isaacs> so... i have a package with lib/foo.js in it. i install it. now how do i get it into my code?
[0:59] <kriskowal> with 1.0, require("foo") as you say
[1:00] <cpojer> it is completely integrated with github -> http://github.com/Guille/APNG this is the current plugin structure (which can, of course, be changed)
[1:00] <isaacs> how does the package installation prog know that the code entry point is <package>/lib/foo.js?
[1:00] <isaacs> just because it looks for lib/<name>.js?
[1:00] <kriskowal> entry point?
[1:00] <kriskowal> as in packages as executables?
[1:01] <ashb> cpojer: ah good your docs are markdown, thats handled already
[1:01] <cpojer> ashb: hm? :D
[1:01] <ashb> cpojer: for my plan for a cpan-a-like of one central place to search for js packages, specifically server side/commonjs
[1:01] <ashb> but probably useful more widley too
[1:01] <cpojer> ashb: very cool, maybe you get some use out of it. It is written in php and symfony though
[1:02] <isaacs> kriskowal: i want to do "npm install ejs". then, in my web app, i want to be able to do require("ejs").parse(someString)
[1:02] <isaacs> kriskowal: in the ejs tarball, there's a lib/ejs.js which is the package entry point.
[1:02] <ashb> the thing isaacs is missing here is the form of requieing a package
[1:02] <ashb> require('package://ejs') or P('ejs').require('ejs') or similar
[1:03] <kriskowal> ashb, don't get too far ahead; we're talking in the context of 1.0
[1:03] <isaacs> actually, i'm kinda ok with being "passed by" so to speak. but i want something functional and useful with the current state of node projects, and i need it working like last week.
[1:03] <ashb> yes, but that sounds like the impression i'm getting - how do you load a module in a package.
[1:03] <cpojer> fetch('myPackage').require('ejs'); :P fun
[1:04] <isaacs> so i'd have to build my own require function.
[1:04] <isaacs> i would rather not do that, though.
[1:04] <ashb> isaacs: by entry point oyu mean sometihng like pythons __init__.poy?
[1:04] <kriskowal> isaacs, currently require("ejs") is the way to go, with 1.0
[1:04] <kriskowal> i am also confused about what you mean by entry point. if you require("foo"), it would be found in the lib/ dir of one of the packages
[1:04] <isaacs> kriskowal: great. so how does the package manager know that it needs to make ejsPackage/lib/ejs.js accessible as "ejs"? put every package's lib folder into require.paths?
[1:05] <isaacs> that sounds like a recipe for craziness.
[1:05] <cpojer> putting everything on require.paths would be crazy slow at some point
[1:05] <isaacs> also, how many packages have a "utils.js"?
[1:05] <kriskowal> that's how narwhal presently works. it does a topological sort of the dependencies to make sure that the search order is sane.
[1:05] <isaacs> yikes.
[1:05] <isaacs> k
[1:05] <isaacs> not gonna do that.
[1:05] <kriskowal> there's also the option of doing overlays
[1:05] <ashb> that being one of the major causes of narwhal's slow startup
[1:05] <kriskowal> you can tar copy the lib dirs onto a single directory
[1:06] <kriskowal> frankly, rhino/java is the cause of narwhal's slow startup
[1:06] <cpojer> isaacs: may I ask what you are developing?
[1:06] <ashb> i thought it was all the stats caused by the package dirs?
[1:06] <kriskowal> i won't say we optimized too early.
[1:06] <isaacs> i'm setting up a proxy module at .node_libraries/foo.js that proxies to .node_libraries/foo/package/lib/foo.js
[1:07] <isaacs> cpojer: http://github.com/isaacs/npm
[1:07] <cpojer> fun
[1:08] <kriskowal> there are more than enough ways to implement it for scalable performance.
[1:08] <kriskowal> it just takes more work on the installer's part.
[1:08] <isaacs> not so much worried about performance as much as implementation complexity.
[1:08] <isaacs> reducing complexity has always led me to performance in the past. i have faith.
[1:08] <kriskowal> i would contend that narwhal's solution is "simple"
[1:08] <isaacs> or at least, close to it.
[1:08] <cpojer> performance does not matter at all if you write clean code
[1:09] <ashb> i agree with isaacs here
[1:09] <isaacs> in this mechanism, i don't have to change any require.paths stuff, and proxying a module is easy with setExports
[1:09] <isaacs> you just need to drop a single file in the path where it'll be found, and then do require("foo") to load whatever foo's author has decided is the best module to expose for the package.
[1:09] <isaacs> if you want something deeper, you can also do require("foo/package/lib/bar") or something
[1:10] <ashb> so you can only require the main module fomr outside a package?
[1:10] <isaacs> ashb: it's just files. so you can require whatever.
[1:10] <kriskowal> i'm not sure what we're disagreeing about
[1:10] <ashb> kriskowal: wether doing a topoligical sort counts as simple i think
[1:10] <isaacs> kriskowal: well... i need to know what that entry-point module is. and i don't see where that exists in the packages.json spec as it is now.
[1:11] <ashb> isaacs: it doesn't cos you didn't bring it up and no one else had that concept in mind
[1:11] <kriskowal> the topo sort is not necessary
[1:11] <kriskowal> unless you have deliberate identifier collisions
[1:12] <isaacs> ashb: you can even require a file from an inactive version, once npm has such a concept, since they'll just be another folder in there.
[1:12] <kriskowal> which you either (a) should not or (b) should use a topo sort to resolve.
[1:12] <ashb> kriskowal: depends how you end up requring packages
[1:13] <ashb> i.e. if you use a flattened namespace approach or a prefixed/url/2 form require approach
[1:13] <ashb> to my mind anyway
[1:13] <isaacs> ok, so... kriskowal you've talked me down from the ledge regarding crazy run-time fetching business.
[1:13] <kriskowal> i would contend that "using-packages" will eventually become the dominant pattern because it is both simple and performant, and can be built on what we've already speced.
[1:13] <ashb> kriskowal: yeah something like it
[1:13] <kriskowal> it's just a bit different than what you'd expect, so it's a good version 2 thing
[1:14] <ashb> bit even there, now that isaacs's mentioned it - i could see a 'main module' for a package having a use
[1:14] <isaacs> and it seems that an entry point module id just never came up. how can we add it? or, if i'm going to start implementing and expecting it, what do you suggest i call it?
[1:14] <isaacs> right now i'm calling it "lib" which is clearly NOT going to work.
[1:14] <kriskowal> we had "main" modules in narwhal for a while. they turned out to not be useful, i think.
[1:14] <ashb> require('package://markdown') == require('package/markdown/markdown') <-- both load lib/markdown.js
[1:14] <ashb> isaacs: 'init' or 'main'
[1:14] <ashb> ?
[1:15] <cpojer> isaacs: why do you actually need to know the name of the main module?
[1:15] <isaacs> cpojer: because i proxy that to a top-level thing that's returned when you do require("blah") where "blah" is the name that you've installed the package to.
[1:15] <cpojer> ah
[1:16] <cpojer> well I wonder if all packages would actually have some sort of main module
[1:16] <cpojer> a package may be just like three files that somehow relate
[1:16] <ashb> a lot would, not all tho
[1:16] <isaacs> cpojer: from what i've seen, most packages seen to
[1:16] <isaacs> and if they don't, it's ok, it's optional
[1:16] <isaacs> you can still do require("foo/package/path/to/whatever")
[1:16] <isaacs> (with npm)
[1:17] <isaacs> it wouldn't be too much harder to extend require() to support putting the "package" before the "foo" like ashb did above
[1:17] <ashb> thats just the convention i've been using
[1:17] <isaacs> so require("package://foo/bar/baz")
[1:17] <cpojer> sorry if those questions may sound stupid, I did not actively follow the packages discussion
[1:17] <ashb> i'm not sure i acutally like it as its quite verbose
[1:17] <isaacs> ashb: agreed.
[1:18] <isaacs> however, as i keep a bit of metadata on each package, besides just the raw package contents, the extra folder is important.
[1:18] <ashb> but it is at least clear what its means
[1:18] <cpojer> I like package://
[1:18] <isaacs> maybe npm://foo/bar/baz
[1:18] <cpojer> even though it could easily also be package:
[1:18] <isaacs> :P
[1:18] <isaacs> package:foo/bar/baz
[1:18] <ashb> there are 2 or three other proposals
[1:18] <cpojer> looks cleaner and is less to type
[1:18] <ashb> one of which is P = require('packages'); P.require('foo','bar')
[1:19] <ashb> or sometrhing approaching those lines
[1:19] <cpojer> that is ugly
[1:19] <ashb> the other major one is 'foo#bar'
[1:19] * isaacs prefers not adding system modules
[1:19] <ashb> isaacs: 'system module'?
[1:19] * isaacs likes paths to look like paths
[1:19] <isaacs> ashb: like, stuff that must be built into the core.
[1:20] <isaacs> but i suppose npm could add that just as easily as anything else outside the core libs.
[1:20] <cpojer> well I consider it as being very unpractical to provide paths if you manually manage packages (not sure if that is something you want to do?)
[1:20] <cpojer> the most portable solution is always to add stuff to require.paths
[1:20] <ashb> yeah. the argument for package manager as a moduel was it doesn't change the behaviour of core require
[1:20] <cpojer> so package: could at some point eliminate the need for it if you can specify require.packages or something
[1:21] <isaacs> ashb: and that's definitely possible.
[1:21] <ashb> http://raphaeljs.com/pie.html
[1:21] <ashb> wow shiny
[1:21] <isaacs> however, wiring up smartly in the require.paths so it Just Works means you don't need to touch the package manager except when you're managing packages, which i prefer.
[1:22] <isaacs> like how pear just drops stuff in your pear dir, and you include it
[1:22] <cpojer> I do not see why you want to introduce another API for packages
[1:22] <isaacs> but you're not dickering around with pear except when you need to
[1:22] <ashb> excpet anything involving require.paths is going to suffer collisions isn't it?
[1:22] <isaacs> ashb: well, see, i don't put much there.
[1:22] <isaacs> just the proxy to the entry point module, and everything else is bundled up in a folder.
[1:23] <isaacs> and at install-time, npm prevents package name collisions
[1:23] <isaacs> (ashb: not just shiny, *jiggly*. i want to eat its delicious gumminess!)
[1:23] <ashb> doing that in hippo would be even easier, we can alias modules by just putting things in require.alias
[1:24] <ashb> i guess its not much difference in behaviour actually
[1:24] <isaacs> yeah
[1:24] <isaacs> the proxy module npm writes looks like this:
[1:24] <isaacs> / generated by npm, please don't touch!
[1:24] <isaacs> module.exports=require("/Users/isaacs/.node_libraries/npm/package/npm");
[1:24] <isaacs> cept that's two /s, of course
[1:24] <ashb> where as for hippo it would be a json file most likely, and then merged into require.alias
[1:25] <isaacs> this was my main impetus for the whole setExports business.
[1:26] <ashb> interesting approach to packages actually.
[1:26] <ashb> anyway - is late, g'night
[1:27] <isaacs> ok, i'm gonna rename my "lib" member to "main" then
[1:27] <isaacs> and start expecting package.json files to look like commonjs package.json files, and ignore what i don't care about.
[1:29] <isaacs> kriskowal ^
[1:33] <isaacs> oh, one more thing, re: packages...
[1:33] <isaacs> what about if i have a make command or something?
[1:34] <isaacs> it's a bit annoying to have to put any setup into js, when it might just be like a "make && sudo make install" or something
[1:36] <kriskowal> isaacs, we haven't adequately addressed the installer issue yet.
[1:36] <kriskowal> that needs to go into a future rev
[1:36] <isaacs> k
[1:36] <isaacs> i'm not happy with what i've implemented so far.
[1:36] <isaacs> i've been spoiled over the last few years by yinst.
[1:36] <kriskowal> we have the "scripts" provision
[1:37] <isaacs> right, is ee that
[1:37] <kriskowal> which is why i'm surprised you say that the spec is too detailed. we really agreed to a bare minimum to get it out of the way
[1:38] <kriskowal> there are informational things, sure, and stuff that not everyone is going to use, but it does support the major base cases
[1:38] <isaacs> well, i'm thinking more about hte imposition it places on the folder structure of packages.
[1:39] <isaacs> it's not so bad if someone already uses it, but "hey, add this file" is way easier than "hey, restructure your project's folders"
[1:40] <kriskowal> right, that's why we made it possible to entirely grandfather an existing package by adding a package.json that rewrites the default layout, via the "directories" property.
[1:40] <kriskowal> i've had to do that in the past.
[1:40] <kriskowal> little things like {"directories": {"lib": "src"}}
[1:40] <kriskowal> so there's convention and configuration, with the former easier than the latter, but support for both
[1:41] <kriskowal> particularly handy for imposing a "package.json" on something which was clearly not designed to, like the google app engine sdk
[1:53] <isaacs> kriskowal: are the "scripts" intended to be included as modules, or run as a cli program?
[1:54] <isaacs> ie, could i do {"scripts":{"install":"make && make install"}}
[1:55] <isaacs> or, perhaps put that in a shell script, and do {"scripts":{"install":"install.sh"}}
[3:12] <kriskowal> isaacs, i'm not exactly sure
[12:39] <hassox> k
[12:39] <hassox> tlrobinson: I'm here
[12:39] <hassox> haha
[12:39] <tlrobinson> ok
[12:39] <hassox> i'm already here ;)
[12:40] <tlrobinson> k
[12:40] <hassox> I know chain code is a little ugly... that's because my js journey has just started really
[12:40] <tlrobinson> it's true it's not the easiest API for writing async apps
[12:40] <hassox> but fundamentally can you see where i'm coming from?
[12:40] <hassox> have you seen the api?
[12:40] <tlrobinson> oh wait
[12:40] <ashb> hmmmm writing tests is boring
[12:40] <hassox> http://github.com/hassox/chain/blob/master/API.textile
[12:41] <hassox> withot all the other justification guff
[12:41] <tlrobinson> oh i thought you were still talking about AJSGI
[12:41] <hassox> oh sorry
[12:41] <hassox> it's neat that you can return a promise
[12:41] <hassox> dude seriosly
[12:41] <hassox> you nearly had me the other day
[12:41] <hassox> it's so neat and clean
[12:41] <tlrobinson> but yeah async JSGI isn't the easiest API, but a 20 line helper object can make it nicer
[12:42] <hassox> sure
[12:42] <tlrobinson> i haven't looked at chain
[12:42] <tlrobinson> let me see
[12:42] <hassox> but it feels like jsgi is ruby moved to js, an evented system
[12:42] <hassox> because it has to move back through every callstack... streaming is problematic, and becomes difficult
[12:42] <tlrobinson> hassox: javascript isn't inherently "evented"
[12:42] <hassox> every implementation I've seen is ;)
[12:43] <hassox> but I'll take your word for it
[12:43] <hassox> I'm very new still
[12:43] <ashb> every impl of what?
[12:43] <tlrobinson> yes the vast majority if js deployments are in browsers
[12:43] <hassox> of javasciript
[12:43] <hassox> right which is where i"ve seen them
[12:43] <ashb> javascript itself is absolutely not evented at all.
[12:43] <hassox> and a little rhino
[12:43] <hassox> which is also evented right?
[12:43] <hassox> ashb: ?
[12:44] <tlrobinson> hassox: no more than "java is evented"
[12:44] <tlrobinson> which it's not
[12:44] <ashb> hassox: the evented comes fomr how its embedded. the langauge itself has no constructs for events
[12:44] <tlrobinson> rhino has threads
[12:44] <hassox> tlrobinson: but it has events too right?
[12:44] <ashb> erlang is natively evented (i think)
[12:44] <keeto> the "eventedness" of js is a byproduct of the DOM.
[12:44] <tlrobinson> not really
[12:44] <ashb> most langauges aren't
[12:45] <hassox> erlang doesn't support event listeners does it?
[12:45] <hassox> anyway
[12:45] <hassox> we're getting off track
[12:45] <ashb> hassox: event listener = receiving a message
[12:45] <ashb> hassox: the first thing you must realise: there is no track
[12:45] <hassox> you guys write stuff to work with rhino and other interperaters
[12:45] <hassox> are you telling me that they're not evented?
[12:45] <tlrobinson> i'm not exactly sure what it means for a *language* to be evented
[12:45] <tlrobinson> events are APIs
[12:46] <tlrobinson> its orthogonal to the language
[12:46] <ashb> tlrobinson: http://en.wikipedia.org/wiki/Erlang_(programming_language)#Concurrency_and_distribution_oriented_language
[12:46] <ashb> the event/message passing semantics are core to erlang
[12:46] <hassox> tlrobinson: sure ok
[12:46] <tlrobinson> ashb: i suppose
[12:46] <ashb> tlrobinson: its about as close as i cant think to natively evented
[12:47] <hassox> tlrobinson: I'm primarily coming into this from a node.js perspective where there are events, and everything is designed to be async
[12:47] <tlrobinson> yes some languages introduce syntax for that sort of thing, if that's what you mean
[12:47] <keeto> ashb: got a minute? I have some questions on zest.
[12:48] <tlrobinson> hassox: yeah i'm familiar with node's philosophy on async ;)
[12:48] <ashb> keeto: sure, here or in #juice?
[12:48] <hassox> tlrobinson: I know you are ;)
[12:48] <hassox> from the doc page though:
[12:48] <hassox> The interface is careful to never buffer entire requests or responses?the user is able to stream data.
[12:49] <tlrobinson> i think it's good there's a project focused on async, but i don't think it's the one true answer to everything w.r.t. javascript on the server
[12:49] <hassox> I've seen some "frameworks" on node buffer the request as the first thing they do
[12:49] <hassox> tlrobinson: agreed
[12:49] <keeto> ashb: oh, no one's in #juice. x) how about in #flusspferd? it's an implementation question so I don't think it's approp here.
[12:49] <hassox> tlrobinson: I have no exp on server js outside of node
[12:49] <tlrobinson> (by that i meant async, and by extension node)
[12:50] <ashb> keeto: sure
[12:50] <ashb> keeto: oh #juicejs sorry, but yeah #flusspferd is fine
[12:50] <hassox> tlrobinson: do you see where I'm coming from though?
[12:50] <hassox> with my concerns
[12:51] <tlrobinson> yeah, i just disagree that the interface should disallow synchronous operation
[12:51] <tlrobinson> async is excellent in some cases, but unnecessary and a burden in others
[12:52] <ashb> ditto
[12:53] <hassox> there's nthing stopping jsgi from being mounted inside chain
[12:53] <hassox> or express
[12:53] <tlrobinson> hassox: also, going back, i don't see any reason an async JSGI interface couldn't avoid buffering anything
[12:54] <tlrobinson> well, except i think it's entirely unnecessary to avoid buffering request headers
[12:54] <hassox> tlrobinson: so if someone is expecting a sync bit of kit to work, but the request isn't buffered, then what?
[12:54] <tlrobinson> you are given an input stream which can be read asychronously or synchronously
[12:55] <tlrobinson> currently we've only defined the synchronous operation (and i don't think we've even really written that down) but there's no reason it couldn't support both if the server supports both
[12:55] <tlrobinson> for io streams, i mean
[12:56] <hassox> so you can access the stream in node syncronously?
[12:56] <tlrobinson> i believe so, with the ".wait()" business
[12:56] <hassox> ryah is talking about removing that
[12:56] <hassox> I really need to get to bed
[12:56] <hassox> but one thing before I go
[12:57] <tlrobinson> as you could probably guess, i think that'd be a mistake
[12:57] <hassox> let say I was iterating over a large array in a browser and doing something with each element
[12:57] <hassox> you might wanna have a chat with him then ;)
[12:57] <hassox> so lets say i was doing said iteration
[12:58] <hassox> to make sure that i didn't freeze the browser
[12:58] <tlrobinson> ok...
[12:58] <hassox> I'd iterate a couple or 1 then do a set timeout in a closure and in the callback do the next one right
[12:58] <hassox> that way I don't hog the event loop and the ui keeps working
[12:59] <ashb> it would be much easier if oyu could just call event_loop.poll();
[12:59] <hassox> that's how I'm viewing htis
[12:59] <hassox> this
[12:59] <hassox> ashb: you see what I mean though right?
[12:59] <hassox> tlrobinson: ?
[12:59] <tlrobinson> yeah i understand
[12:59] <hassox> do you see where I'm coming from?
[12:59] <hassox> so that's what I'm trying to avoid
[13:00] <hassox> if it's simple to make that happen, then why not?
[13:01] <tlrobinson> isn't that exactly why they introduced web workers in HTML5 ;)
[13:01] <tlrobinson> it's a pain to keep track of that sort of thing
[13:02] <hassox> Workers (as these background scripts are called herein) are relatively heavy-weight, and are not intended to be used in large numbers
[13:02] <ondras> why?
[13:02] <hassox> I like the idea of not resorting to threads
[13:02] <hassox> ondras: I don't know... that's just what it says in the spec ;)
[13:02] <hassox> http://www.whatwg.org/specs/web-workers/current-work/#introduction
[13:03] <ondras> okay :)
[13:03] <hassox> sorry should have quoted
[13:03] <tlrobinson> but you'd only need one to handle that heavy workload without blocking the ui
[13:03] <hassox> sure
[13:03] <ondras> hmm
[13:03] <hassox> but when we're talking about many connections to a web server, and potentially out of request cycle processing sending mail and stuff
[13:04] <hassox> it ammounts to many of them right?
[13:04] <tlrobinson> yes
[13:04] <tlrobinson> it's a tradeoff
[13:04] <tlrobinson> imo it's harder to program async APIs
[13:04] <hassox> tlrobinson: agreed
[13:05] <hassox> it is harder
[13:05] <ashb> i think most apis should be opaquely async
[13:05] <hassox> but not impossible
[13:05] <ashb> http://pastie.org/776373
[13:05] <inimino> creating a worker is somewhat expensive, dispatching work to one is not
[13:05] <inimino> I don't see any reason why it has to be harder at all.
[13:05] <hassox> ashb: that blocks right?
[13:05] <hassox> ashb: waiting for the result?
[13:05] <hassox> oh no
[13:05] <hassox> soz
[13:05] <hassox> I see
[13:05] <ashb> hassox: yes but no :)
[13:05] <ashb> while (!done) event_loop.run_once()
[13:05] <hassox> it's getting very late here ;)
[13:06] <ashb> it conceptually blocks, but doesn't
[13:06] <hassox> I'm quite tired but this is so interesting ;)
[13:06] <ashb> yeah i get that a lot
[13:06] <inimino> I just think we haven't put in the effort required to make it as easy yet
[13:06] <tlrobinson> we'll all be here tomorrow, most likely :)
[13:06] <hassox> inimino: workers are threads though right
[13:06] <tlrobinson> anyone going to the SF JS meetup tomorrow, btw?
[13:06] <hassox> tlrobinson: yeah but I'll be at work while you're up
[13:07] <tlrobinson> crap it's 5am here. i should probably go to sleep
[13:07] <ashb> haha. yes
[13:07] <inimino> hassox: workers are not what most people would think of as threads (certainly not POSIX threads)
[13:07] <inimino> hassox: but in a looser sense, sure
[13:07] <hassox> haha
[13:07] <hassox> oh
[13:07] <hassox> sorry
[13:07] <hassox> I read that a loser
[13:07] <hassox> ;)
[13:07] <hassox> I told you it was late
[13:07] <hassox> tlrobinson: you beat me
[13:08] <hassox> but i havae to be up in 6 hours
[13:08] <tlrobinson> hassox: what time zone are you
[13:08] <hassox> australia
[13:08] <inimino> heh
[13:08] <hassox> AEST
[13:08] <hassox> inimino: so what are they?
[13:08] <hassox> do they share state with what's happening in the main event loop?
[13:09] <inimino> hassox: no
[13:09] <inimino> message passing only
[13:09] <inimino> you can send messages to them, they can send messages back
[13:09] <inimino> the messages are self-contained, just like in Erlang
[13:09] <inimino> and there is no shared state
[13:09] <hassox> oh cool :)
[13:09] <hassox> and you can pass full objects including listeners and functions?
[13:10] <inimino> no
[13:10] <inimino> only things that you would think of as "pure data"
[13:10] <hassox> right
[13:11] <tlrobinson> inimino: isn't there some way to pass ports or something
[13:11] <hassox> it would be hawt if there was a way to pass connections
[13:11] <tlrobinson> for other workers
[13:11] <inimino> numbers, strings, structures of these
[13:11] <inimino> yes
[13:12] <inimino> yes
[13:13] <hassox> inimino: as in... yes it would be cool, or yes you can?
[13:14] <inimino> yes you can send a port
[13:14] <tlrobinson> void postMessage(in any message, in optional MessagePortArray ports);
[13:14] <hassox> inimino: so when I recieve a request in node, I could fire up a child process and pass the connection to the child?
[13:14] <ashb> you can do it in posix and win32 certainly
[13:15] <tlrobinson> ashb: we're talking about web workers
[13:15] <tlrobinson> oh nm
[13:15] <ashb> k
[13:16] <hassox> ashb: I'm interested in passing connections about
[13:16] <inimino> hassox: you'd either use a worker that already exists, or create a new one
[13:16] <inimino> hassox: the worker implementation would worry about whether that means a child process, OS thread, green thread, or whatever
[13:17] <hassox> inimino: but you can pass connections about between os processes?
[13:17] * hassox isn't much of an os person :'(
[13:17] <tlrobinson> sendmsg in posix, i think
[13:17] <ashb> yeah
[13:18] <ashb> or something like it
[13:18] * tlrobinson has no idea how that works
[13:18] <ashb> hassox: yes you can.
[13:18] <inimino> ah, if you used different processes to implement it, yeah
[13:18] <hassox> ashb: that is so awesome
[13:18] <hassox> so very freaking awesome
[13:19] <ashb> WSADuplicateSocket on windows
[13:19] <ashb> and...
[13:19] <hassox> it has to be on the same OS instance though right? or can you pass them between machines also?
[13:20] <ashb> something on posix
[13:20] <ashb> no same host
[13:21] <hassox> eyah
[13:21] <hassox> still awesome though
[13:21] <inimino> of course whatever kind of cross-machine IPC you're doing can certainly send whatever it uses to identify other processes on other hosts across
[13:21] <ashb> ah its sendmesg over a unix domain socket
[13:22] <inimino> so you could quite easily have Workers supported with the same API transparently across different hosts
[13:22] <ashb> you'd have to proxy the data back and forth somewher tho
[13:22] <ashb> since you can't pass the actual connection back and forth
[13:23] <hassox> ashb: same host is still pretty hawt all the same
[13:26] <hassox> thanx for the chat lads :D
[13:26] <hassox> it's been very interesting
[13:30] <hassox> gnight
[13:31] <inimino> 'night hassox
[14:04] <ashb> hmmm my test suite/runner stuff could really benefit from setup/teardown
[14:21] <SubtleGradient> gotta have setup/teardown. Both before/after each test and before/after each group of tests
[14:21] <SubtleGradient> optional and separate ofc
[14:22] <SubtleGradient> JSSpec was great back in the day. Now I'm a fan of Jasmine.
[14:22] <SubtleGradient> ashb: ?
[14:23] <ashb> i've not found (m)any JS test suites i like
[14:23] <SubtleGradient> it's true
[14:23] <SubtleGradient> there are far too many in use right now
[14:23] <SubtleGradient> I'd like to see some consolidation
[14:24] <SubtleGradient> but there are some seriously big things that I hate about some of them. And people apparently like them.
[14:24] <ashb> so there is a basica testing spec
[14:24] <ashb> but thats jsut the assertions
[14:24] <ashb> doesn't cover the rest of the stuff yet
[14:24] <SubtleGradient> the JS universe is far too fractured
[14:24] <ashb> yeah - which is where i hope http://packag.es will help
[14:24] <SubtleGradient> i am very interested in that
[14:25] <cpojer> the problem with jasmine is that it is still browser centrict
[14:25] <cpojer> -t
[14:25] <SubtleGradient> i wrote up some basic specs for how I would want something like that to work
[14:26] <ashb> SubtleGradient: we've submitted a talk about it to jsconf.us
[14:26] <SubtleGradient> basically a rubygems or yum for js
[14:26] <ashb> so we'll have to do something for it now
[14:26] <ashb> so to start with its just going to give searching/rendering of docs
[14:26] <SubtleGradient> cpojer: is it? I thought they ran in rhino too?
[14:26] <ashb> i'll try to get it tied intop package managers/installers as they appear
[14:26] <cpojer> only if window is defined
[14:26] <cpojer> and setTimeout etc.
[14:27] <SubtleGradient> cpojer: wow, lame
[14:27] <cpojer> SubtleGradient: fabio and I hacked it, my version of jasmine works in v8cgi too
[14:27] <cpojer> probably node too, did not test it there yet
[14:28] <SubtleGradient> we could maintain a fork, or we could try to help push the entire project towards being very useful for ssjs and cli js too
[14:29] <cpojer> yes
[14:29] <cpojer> probably :)
[14:29] <cpojer> the jasmine API is very, very cool
[14:30] <SubtleGradient> ya, it's like JSSpec, but a bit better
[14:30] <SubtleGradient> and does more ofc
[14:30] <cpojer> no, a lot better
[14:30] <cpojer> expect(a).toThrow(..) reads nicely
[14:30] <SubtleGradient> I'd actually recommend making Jasmine itself much leaner, and moving all the async stuff and Spy and whatnot to an "extras" layer or optional modules or something
[14:31] <cpojer> better than value_of(a).should_be(b)
[14:31] <SubtleGradient> ya, I like that better than `value_of().should_be()`
[14:31] <cpojer> I thought about it openly and people on the mootools dev channel were like "please don't fork it"
[14:32] <SubtleGradient> I'm much more interested in pushing the main project itself
[14:32] <SubtleGradient> I don't want to use code maintained be a single person again
[14:32] <cpojer> well it could always happen that they do not like the changes
[14:32] <cpojer> the minute they don't pull it on github its a fork
[14:33] <SubtleGradient> obviously, which is why you'd get involved with the other devs and propose the change in direction.
[14:34] <SubtleGradient> if they're not interested in going in that direction then we should look for another project with traction that will do what we want
[14:35] <cpojer> maybe
[14:35] <SubtleGradient> but I'm not interested in having an official mootools specs framework, we have more interesting code to write
[16:04] <Wes-> Anybody else implemented fs-base besides me and the flusspferd folks?
[16:05] * ondras will do that once he implements binary/?
[16:05] * ondras will start implementing binary/? once he is no longer ill :/
[16:09] <ashb> ondras: you should just implement a v8 engine for flusspferd, it would be much easier that way >_>
[16:09] <Wes-> ondras: you should just implement a gpsee-like FFI layer for flusspferd, it would be much easier that way >_>
[16:10] <Wes-> er, not flusspferd, v8cgi
[16:10] <Wes-> lol
[16:11] <ondras> ashb: actually, I do not like the name, so this is not going to happen anytime soon :/
[16:11] <ondras> Wes-: actually, I m not familiar with FFI, so this is not going to happen anytime soon :/
[16:11] <Wes-> ondras: I have lots of code to share! But check this out:
[16:12] <Wes-> http://code.google.com/p/gpsee/source/browse/modules/fs-base/fs-base_module.js
[16:12] <Wes-> it is my current implementation of fs-base
[16:12] <Wes-> line 226 is a pretty good example
[16:14] * Wes- spots a bug! % 1000, whoops!
[16:19] <ondras> Wes-: nice, indeed
[16:20] * ondras often wanders if modules should have js+native parts, or only native parts
[16:20] <ondras> (where native support is necessary)
[16:20] <Wes-> ondras: both gpsee and flusspferd allow "blended" modules, which can have both native and js parts
[16:20] <ondras> yes, I see :)
[16:20] <Wes-> in fact, binary/b is implemented like that over here
[16:20] <ondras> but allow != encourage
[16:21] <Wes-> ondras: technically, fs-base_module.js is a pure JS module -- it calls into a general-purpose FFI module for the native code
[16:21] <Wes-> For system calls, my FFI layer (theoretically) portable to anything which qualifies as UNIX<tm> without requiring JS-land knowledge or code changes
[16:22] <Wes-> And Linux is close enough to UNIX<tm> that we make it work, too
[16:22] <ondras> :)
[16:22] <ondras> Wes-: what do you think; is there going to be someone stating that pure native fs-base would have been quicker?
[16:23] * ondras is not qualified enough to judge this
[16:23] <Wes-> ondras: BTW, encouraging mixed-language modules is not a bad thing. Some things are just better in JS
[16:23] <Wes-> ondras: I doubt that. I rolled fs-base is about 1.5 days
[16:23] <Wes-> I am confident it would have taken me ~7 in C. Maybe more.
[16:23] <ondras> eh
[16:23] <ondras> performance wise
[16:24] <Wes-> Well, performance wise -- this is where you measure and pick which functions to optimize. :)
[16:24] <ondras> I fully agree that many things are better in JS
[16:24] <Wes-> That said, look at the code -- there is not much JS there! Most of the time will be spent in relatively expensive systems calls, I would guess. (Note that guess != measure. :) )
[16:25] <ondras> and your ffi-based approch looks truly sexy
[16:25] <ondras> perhaps I will try to learn something about ffi now
[16:25] <Wes-> I am quite pleased with it. :)
[16:25] <ondras> .)
[16:25] <ondras> *approach
[16:25] <Wes-> You COULD try to port my ffi layer to v8 - it's probably not all that hard. It's reasonably well encapsulated.
[16:25] <ondras> Wes-: just tell me - how do you store file pointers?
[16:26] <Wes-> ondras: take a look at line 110
[16:27] <ondras> hm
[16:27] <ondras> that is because fd is a number
[16:29] <ondras> hm
[16:30] <ondras> I am still not sure how passing around return value of _fdopen() works :)
[16:31] <Wes-> ondras: fdopen() returns a FILE *
[16:32] <ondras> and that is in some magical way converted to a js value
[16:32] <Wes-> ondras: I store both the fd and the resultant FILE *; I may need one or the other depending on the resultant Stream type
[16:32] <Wes-> ondras: part of my FFI layer takes care of that
[16:32] <ondras> ah
[16:32] <ondras> so, what exactly in JS world is a FILE*?
[16:32] <Wes-> ondras: notice also the finalizeWith() call
[16:33] <Wes-> ondras: It's a JS object which stores a pointer in a special hidden slot; whenever any FFI call uses this object and needs a pointer, it knows to go digging in this slot and to pull the pointer back out
[16:33] <ondras> ah
[16:33] <ondras> hm, that is doable in v8 pretty easy; basically in the same way
[16:34] <ondras> Wes-: btw, those js+native modules; how are they treated by require() ?
[16:34] <Wes-> Yeah, I imagine pretty everything will translate. finalizeWith() might be tricky, you need to be able to hook into the garbage collector
[16:34] <ondras> I am able to be notified when a GC of a given value happens
[16:34] <Wes-> ondras: the native module runs first, attaching properties and methods to a specific JSOBject. Then that object is passed to the native code as the exports variable.
[16:35] <Wes-> ondras: that's all that's needed. This is totally implementable in v8, I bet
[16:36] <Wes-> You can also implement "catch-all" properties, right? That's needed for structs
[16:37] <ondras> catch-all?
[16:37] <ondras> you mean obj.whatever -> execute a c code?
[16:48] <Wes-> ondras: Yes, for arbitrary property lookup (getter/setter)
[21:27] <jmerlin> ondras: in v8cgi, how are you keeping loaded modules from polluting the global object?

 

 

Logs by date :